Process Modelling and Process Oriented Organisations

I have recently been on a course about Business Process Modelling, which gave me some other perspectives of BPM and how it can be promoted and used by companies.

Prior to the course, I have used a few BPM tools to graphically represent business processes (including IBM Business Process Modeller, Oracle SOA Suite and AquaLogic BPM tools). When using those, I have felt that their use is more related to the automated execution of business processes, through BPEL and service integration capabilities. This seems to be just a technical approach, which isn’t really the solution – although that does depend on how you classify the problem to be solved.

In working with clients, and team members, I have had numerous discussions and explanations on modelling of business processes where there is no technical view involved. It is from this approach that modelling the process can help in understanding, defining and subsequently refining or improving the operations of an enterprise. In process modelling terms, this would consist of creating the Process Architecture for an organisation. From an Enterprise Architecture, this could correspond to the Business Architecture view, where the business is represented as a series of processes operating across defined value chains.

The interesting part of the BPM course I attended was not so much around the approach and methodology described for modelling, but more about the real-world experiences, cases and examples put forward by the course trainer (who does this as a real job, not just as a trainer). Many industry speakers and research analysts are promoting the benefits of process modelling and in structuring companies around process delivery and value chains, rather than in the more common functional silos and hierarchies. The examples discussed on the course seemed to put forward a strong case for this approach (as you would expect), but perhaps with more explanation. My task could now be to summarise and promote those concepts within my own company, as well as that of clients that we work with.

A parallel was frequently drawn with the way in which manufacturing has long been structured and organised around processes, with the production line of Henry Ford being one example. Also, the quality and time controls from Deming et al being another way in which that industry has rationalised and improved over time.

In contrast, the service industry is still mostly structured in functional areas, with a process crossing over many areas and responsibilities from start to finish. That is clearly inefficient. The hand-offs between teams mean that no one person or function is responsible for the process as a whole; each functional area is mostly tasked with ‘doing their job’ and not focused on the end result of the process within which they are working. This is further highlighted by the way in which company budgets are defined. These are nearly always on an area-by-area basis, not on a process-by-process. That can result in functional areas not having any strong interest in delivering value through a complete process, merely in getting their piece of work done in the quickest, easiest or cheapest way. Generally, not the behaviour that an organisation would truly wish for.

A point I would like to repeat from the course is that processes should have objectives or targets and associated measures that cover not only efficiency, but also effectiveness and adaptability. Many measures look just at basic efficiency, which is a waste if the overall process being followed is not actually effective (why optimise a part of a process to the nth degree if that part isn’t even needed in the first place). Analysing and designing processes against such measures can give an objective outcome for any recommendations, and a way to show the gains as a result. Consider also that efficiency gains through optimising process steps may gain percentage improvements, but making larger-scale changes to the overall process may result in many-fold improvements. Figures such as reducing a process from weeks to hours, those are the major gains available to companies if they do things right.

I would think that the one take-away from my views and comments above should be that organisations can definitely be improved through process re-engineering, and that business process modelling is a great approach to achieve this in a visible way. In fact, if you spend time looking at things from a BPM perspective, you would wonder how companies can look for large benefits without such modelling and associated organisation structures.

 

Business Transaction Handling

Have had discussions recently with various colleagues and clients about the way in which business-level transactions need to be implemented in a reliable way that also supports cancelling the actions that have been performed if and when something goes wrong in the process.

When using technology, the use of XA-compliant transaction managers and two-phase-commits give much of what is required. However, those facilities are not always available, and even when present they only cover a particular set of steps or actions in an overall process.

It is important to take a higher-level view of the operations being performed by the associated business process (not only BPEL and BPM, but the real business process, involving people and actions).

The need to unwind or roll back an operation, or sequence of operations, is not only down to technical updates in a database, but has to consider the implications this has on the business.

Where a plain rollback is not possible, there is a need to perform some form of ‘compensating transaction’ to correct the erroneous actions. This can be quite complex, depending on what update(s) went first, what auditing is required, how the updated data might already have been used, and what actions should follow the ‘corrective action’. In an SOA environment, this logic is also to be applied to the definition and implementation of services – if an ‘update’ service is provided, consider if there is a need to provide a related ‘revert update’ service that records the necessary information.

An architectural principle may be defined to state that application updates should not require compensating transactions, but in reality there will often be a need to do some form of update and change of processing.

Caution is required to avoid only performing basic updates to reverse the original transaction. From an accountancy perspective, this is often shown by a balancing transaction that negates the original entry, rather than removal of that older entry. Applications should correct data in the same way, making it easier to identify the sequence of events of the overall business process.

Looking again at the technical aspects, the implementation of multi-stage transactions begins with transaction handlers, which normally exist within the application framework or container (or the operating system itself). As more and more steps occur within a process up to the point at which a failure may arise, this requires additional routes for error handling to unwind the corresponding transactions and notifications.

This may be implemented using specific code in an application, but this may bring problems of its own in complexity and reliability, with maintenance also being made more difficult should any of the steps be changed. We have looked at how this error handling is implemented by the BPM tooling of WebSphere Process Server, and this does provide a clean way to applying compensating transactions based on the logic applied before identification of an error. That does seem to be a great benefit for complex business processes with automation in this way, but it isn’t exactly cheap if considered just as a way to wrap up error handling.

Anyway, discussions come up with various thoughts and recommendations as to how best this is solved for different situations. The most important thing to note is that analysis and design of business processes MUST address the approach taken to transaction handling in error conditions.

 

Agile methods and BPM

Here’s a question for you – do agile methods and BPM go hand-in-hand? I heard the other day of a project adopting an agile approach to developing automated complex back-office business processes – and at risk of falling into the trap of delivering each use case with a heavy focus on the presentation layer but not really designing/thinking about (and presumably refactoring) the business process layer. Most important is refactoring fledgling process functionality out of the presentation layer into the BPM layer.

I’m sure it can be done – but I suspect that the chances of success are significantly lower using agile methods than for a more classic pure web development. Thoughts anyone?