Blog Posts Process Analysis Process Modeling

Most Common Questions on Implementing BPM + SOA

Blog: Process-Modeling.com - Rick Geneva

Recently I have been doing a series of presentations on SOA and BPM with a combined governance strategy and framework. It seems that BPM and SOA are a hot topic these days, but there doesn’t seem to be much knowledge on how to effectively combine both practices into a unified effort where both IT and business collaborate towards the same goals. This is the problem that BPM tried to solve back in 2002 but was not widely adopted because of the lack of IT backing of the tools. Now that SOA is becoming common practice it’s time for a second look at what BPM can really do for an organization.

Included in this tour: Atlanta, Chicago, Vancouver, and then wrapping it up in San Francisco. The audience was fairly broad, including project managers, IT engineers/architects, business analysts, and even a few executives. There are several things I found in common from the questions at each of these sessions.

1. The IT trends are always changing, so why should I start now with BPM and SOA?

That’s a great point. Interestingly I get this question every time I present this topic. Yes, technology trends are always changing. But this is something different. Previously the right elements were not all in place to fully take advantage of the BPM paradigm. Today many BPM software vendors are creating SOA based products, and SOA software vendors are creating “orchestration” products which are very similar to BPM concepts. Basically out-of-the-box you get a whole lot of BPM and SOA from one product. Previously (2002 – 2005) this would have required several layers of middleware and custom connectors to achieve. So the best way to say this is that now BPM and SOA are consumable by an organization. It’s not quite to the point yet where the systems are such a commodity that it’s like a word processor. But even the common word processor is starting to have some collaboration capability built in. So where do you think this is going? In my opinion I think we are past the point of experimenting with BPM and SOA. It’s now become common practice in many organizations and has enough financial backing due to the customer demand that it’s here to stay.

While in my Vancouver session one of my attendees asked this same question in a way that I found interesting. Their organization had worked with various case tools and workflow automation tools over the years and they said that they have seen many “code generation” tools that have made the same claims as BPM promises to deliver but they never did. So it seems that the motivation for the skepticism is based on slick sales people offering that classic “80% out of the box solution” that you can “fire your IT people and do it yourself”. Well, you’ll never hear me telling you to fire your IT people. You might need to educate them on process modeling, but you still need them to create good SOA components for you.

The BPM vendors of the early decade (2002 – 2005) are partially the cause of the skepticism of the BPM potential. It used to be a hot buzzword back in 2003 but it went away for a while before coming back strong in 2008. Often a vendor would try to sell BPM to an organization. Who would have known that you have to actually do BPM, and you can’t actually buy it? Well, times have changed and it’s time to take a second look at what’s available.

Another reason why I see it’s time to get into BPM and SOA is that organizations tend to look at ways of being more efficient in times where cash is tight. In times of plenty, we just hire more warm bodies to throw at a problem. Later we can just fire them, right? Or on the other hand, we can learn to manage our processes and systems in a more efficient way so that we don’t have the huge downfall after times of rapid growth. After a period of recession there will be a period of growth. After you’ve already grown out of control it’s hard to get a handle on the problem. So maybe you can get started now before you miss this window of opportunity.

2. What are the most common problems I see in implementing BPM?

This one is easy. There is not a lot of BPMN process modeling skills in most organizations. The people that I find who know how to model a process well are usually consultants such as myself. A basic BPMN training course is a good start. But after the basics of BPMN are learned additional coaching/consulting is likely needed to do a full scale project. Process modeling for the purposes of automation/execution is much different then modeling the ‘as-is’ process. Different patterns are used, and sometimes these patterns are vendor specific for the BPM automation system. So it’s one thing to model your ‘as-is’ process. Learning how to make it actually work is another skill that you must learn by practice.

So the short answer to the question of what are implementation challenges is that there is no widely practiced methodology that works well with both SOA and BPM. Waterfall? Forget it. Scrum, Agile? Sort of, but not quite. Test driven? Mmm.. partially. PMF includes the best of all the above, and is totally oriented towards the business objectives and process flow. On one hand you have methodologies that are totally focused on data models and on the other you have high-level flowcharts. In the middle there is really nothing to bridge the gap.

For this reason I have created the Process Modeling Framework (PMF). PMF was briefly introduced in my book The Microguide to Process Modeling in BPMN but we did it no justice. The book’s title says “microguide” so please don’t’ be disappointed if the concept is a bit thin in this book. The intention of the book is to give you a quick start on a consumable, non-technical format. But for those of you who want to learn more I’m in the process of writing the next book that will cover PMF in great detail.

3. What are the most common challenges I see in implementing SOA?

Most of the time I see engineers creating data models as the first step to a system design. This I see as the greatest roadblock. What you have to understand is that data would not exist if there was no process first. What is the point of having data? Do you just create data as the objective of the process? Or is there a reason for doing so? In other words, stop modeling data and start modeling processes. The process will dictate what the data requirements are, and not the other way around.

I realize this is nearly treason for some of you on the SOA side of the fence. To the more technically minded the data is everything. And I totally agree with you that data is important. But let’s not get into a “chicken or the egg” debate. I you already have data, manage it, but in a way that supports the business objective. Don’t start going out of your way to create an SOA infrastructure when you don’t even have a solid process model to back your assumptions. Otherwise you might end up making the situation worse by creating an architecture that restricts the organization’s capability to grow.

4. How do I handle developing many services and many subprocesses in parallel?

For the most part you need to limit the scope of diagrams to a single business objective. Stop creating flow charts that wrap around the room. Stop modeling everything in terms of swimlanes. There is more going on here than one person passing a task to another person. Any process in a large organization likely interescts with other processes. So maybe what you should be drawing is how the processes interect with each other, and not how people interact. People are part of the organization, and the organization is made up of processes. This all relevant when determining what is the scope of a diagram. Once you start mixing objectives from processes that overlap/intersect/conflict with each other, then there is no way you can have multiple people work on it. It’s like saying that you are going to take 10 race car drivers, give one the steeing wheel, another to the brakes, another to the acellerator, etc. It’s just not going to work this way. Every driver needs to have their own car. So stop trying to model everything as if there is only one car with many lanes (swimlanes). Instead there are many cars, with many lanes, and not all of them flow in the same direction.

5. How much of the BPM/SOA methodology should I teach to the business stakeholders?

At first, as little as possible while still allowing yourself to do your work. Only explain as much as what is necessary to get the information you need from the stakeholders. Otherwise you’ll get into nearly religious debates about methodologies that evolved from the 1970’s. Gradually as you show results, the organization will adopt BPM/SOA because of the obvious gains in productivity, visibility accountability, and agility. So instead of upsetting the entire organization overnight, start slowly and show results, not radical concepts. Results in process modeling come from showing how to improve the organization. Results in SOA come from showing how quickly you can create a composite system from several existing components.

Today we have accelerated expectations yet we still use methodologies based on a time when computers were the size of refrigerators and access to a computer was only for those lucky enough to have one in their office. Yes, you are trying to overcome this problem. But you are fighting an uphill battle with “traditional wisdom”. Traditional wisdom doesn’t have a 3 GHz quad core CPU with 8 GB of RAM, connected to the internet at 10 gigabits per second. But I do, and you probably do too. Computers are just as much a part of the organization as its people are. 90% of corporate computer systems are sitting idle 90% of the time because we still haven’t figured out to efficiently use them yet as part of our business process. We’ve got way more computing power than we know what to do with, yet we still pass around pieces of paper or spreadsheets over email.

Let me ask you this: When is the last time you emailed a spreadsheet to a group of people and collected responses? Sorry, but this is not a process. It’s chaos in the making. Yet a majority of the organizations I work with use this technique of information mis-management as common practice. They know there is a better way, but at the same time they resist changing. After all, if it isn’t broke, why try to fix it, right? I’ve got a reason of you. It’s called financial meltdown – just incase you haven’t been watching the news lately.

The older methodologies never took this phenomenon of high speed computing into consideration because it didn’t exist yet. The world has changed dramatically in the past decade. Now I can connect to people via my “video phone” in a virtual world and I can book my own travel tickets faster than I can explain to someone where I want to go. Or better yet, I can schedule a trip to Chicago on my calendar and my saved preferences will automatically book my flight for me, and all I have to do is confirm it. This is an example of process oriented, as opposed to the way it’s always been done.

You have to consider that you cannot change the mindset of “the machine” of your large organization overnight. Some very respected people work in this same organization as you have built their reputation on these older methodologies. One tiny voice shouting BPM and SOA will not be heard over a hundred entrenched opinions based on traditions. By going on a crusade to change everything, all you are doing is shaking the tree of traditional wisdom, and whatever falls out of this tree is likely to come down hit you hard on the head. Many of these very respected people still call a travel agent over the phone and use a paper ticket at the airport. On the other hand, many of them ride around in a corporate jet and haven’t seen an airport security line since 2000 when it was still okay to meet your friends as they walk off the plane at the gate.

If you are lucky enough to have a CIO or CEO that mandates the organization will have a BPM/SOA center of excellence team (CoPE team) then you can go ahead and start a formal training program. But still, the details will be missed by most people in the organization. People tend to do 5 to 7 activities in their processes. Outside the scope of what they do on a day to day basis is hard to grasp. The bigger picture of a formal BPM/SOA practice is likely to be well received to a C-level executive, a VP, a director, etc. But anyone lower than this and it’s likely they’ll need a Business Analyst or IT architecture background to understand where you are going with these concepts.

But the good news is that over time as the organization transforms its self, every one else will eventually understand by seeing examples of good practices in action. I’ve seen even the people that have the hardest time changing eventually learn the process oriented approach. It’s contagous. And once you go process oriented, you’ll never go back.