WSO2 ESB – Why this can’t be ignored…

I have been asked many times to help our customers select a product to act as their ESB and have constantly found myself resorting to type and looking at the usual suspects IBM, Oracle, Red Hat, MuleSoft etc. However, recently I have been looking in greater detail at the ESB offering from WSO2 especially since the coverage and ratings from Gartner and Forrester, so when I was  asked by Smart421 to attend the WSO2 ESB and Enterprise Integration last Tuesday (6th September) I jumped at the chance.

Arriving at the IET in Savoy Place, London I was greeted by that fantastic smell of fresh coffee first thing in the morning, that gave me opportunity to speak to some of the others attending, representatives from SpecSavers, Aspen Re, SportingBet there to find out more about WSO2 and the products they offer, to other systems integrators like Asteria. Paul Freemantle (CTO of WSO2) was leading the workshop and I enjoyed the openness of him and the easiness of his style of presenting which I am sure is driven from the passion he has for WSO2 and the products they are building.

What struck me most about the WSO2 ESB during the workshop and demonstrations was the simplicity of  it WSO2 ESB is a lean, lightweight mediation platform based on and enhances the Apache Synapse ESB and has been designed for the demands of high-volume SOA implementations. Built on the OSGi specification, WSO2 ESB can be easily customised to specific IT project needs by adding other WSO2 components , providing greater flexibility and agility to meet changing and challenging enterprise demands. WSO2 ESB is also available as an ESB-as-a-Service as part of the WSO2 Stratos cloud middleware platform and WSO2 StratosLive platform-as-a-service (PaaS) hosted by WSO2 again making an even stronger case for adopting this.

Paul also talked through some of the examples of clients using their products such as British Airways, US NavyMicrosoft and eBay – with their ESB processing over a billion transactions per day! Couple this with with what had been demonstrated during the day and the other products that WSO2 offer such as they Governance Registry, Application Server and Business Process Server you can see the capability that you have available to you immediately. You can see why this has helped Gartner and Forrester rating this product so highly, and why I think that this is a serious option when considering an ESB and one I will be pushing more…

 

Open Source Middleware (a.k.a. ESB) Choices – Part 1

In part 1 I set out my requirements for evaluating and choosing an open source ESB, and promptly fall down a rabbit hole of OSGI and Maven when I get to see how much more there is to them than I was previously aware.

From time to time we get requests on how to get started with middle-ware technology on the cheap. Here the emphasis is just about connecting service providers and consumers up, without getting into anything fancy like orchestration or service repositories. Of course, in an ideal world the solution should not rule the latter out.
So here are the requirements I can filter out of a few of these conversations.

• Open source for low cost and wider options when upgrades appear – i.e. not always forced onto the latest version.
• Must handle WS-Security, WS-Addressing
• Freedom for choosing java-XML binding framework
• Supports contract-first services design (as opposed to generating the service artefacts (WSDLs, schemata) from java classes.
• Run-time is ‘light’: i.e. when service-enabling components are deployed on the same machine as an application which will be service enabled, these service-enabling components do not gobble up all of the resources.

Contract first development is very important in a heterogenous environment. See arguments on the object – XML impedance mismatch here. Another way of putting it is: if you are going to stick with just one language (e.g. java) then why bother with XML in the first place – just go with some RMI technology, RMI-IIOP. If we are using Web-services, then interoperability is a big consideration, and for that we have to think contract-first.
One of the reasons for separating out the java-XML binding from the web-service end-point binding code is that it is great to use the same pojo to describe an entity, whether it is serialised to XML or persisted to a database or just as a value object.
On the one hand it is good practice to work with web-services in a contract-first style, and on the other hand: if you use the code (specifically the domain objects to which the XML is bound) throughout your application then you can introduce a dependancy on the XML marshalling generated classes, which is not great either. In an automated build environment, it means building your gen-src directory from schema before you can write any src code which uses it.
In the past I have got around this by specifically generating class libraries of domain objects, using jaxb, and then importing the resulting jar into any code (both client and server side) which manipulated these objects. The compromise at the time was that I ended up writing my own endpoints (servlets) to expose web-services – which is OK when there is not much WS-*, (e.g. Addressing, Security) going on.

I wanted to see if the latest incarnation of the open source frameworks would enable contract first, generation of domain objects first (such that they could also be used in persistence layers and as value objects) and relatively easy handling of WS-Security and WS-Addressing.
The new kids on the block for me are ServiceMix (a.k.a. Fuse), apache cxf, spring-ws, and the sun JAX-WS.

The previous time, the players had been Apache Axis, WSIF, and JAX-RPC. Oh I almost forgot Castor. Every one of these had had their own java-XML binding code and none of the produced beans were interoperable between frameworks. Stand-alone java-XML binding frameworks like JAXB (1.x) were not interoperable with the web services binding (e.g. JAX-RPC) generated objects.

Anyway: enough of the background… The first two I wanted to look at were both FUSE and Spring-WS, as they both allow contract first (Spring-WS will not allow anything else) and they both support Spring and its IoC (another Good Thing, but that’s a different discussion).

I had only got around to looking at FUSE when I fell down the first rabbit-hole: OSGI and Maven. I have had a look at the excellent video tutorials by Adrian Trenaman of progress (formerly Iona) software, (see the demo videos tab at the bottom of this page.
I had been aware of Maven for a while as a ‘slightly better Ant’, but the demo and a bit more digging around reveals there are two big extra features in Maven which move the game on a whole lot more:
Firstly there is the project template feature. This is the feature whereby you can create a java project using a Maven build file with a command-line argument. The command builds the appropriate directory structure and even the correct POM.xml (Maven equivalent of an Ant build.xml file). Although I had been demo’d this before, it has only really sunk in this time what a big deal this is.

We have in the past put a lot of energy into our automated build system, based around Ant. For it to work well, there is a mandated project directory structure, and set of files and libraries have to be in the right places relative to each other. There’s a bit of a learning curve on top of Ant to understand what is going on. The template projects from Maven give you all that in one go. That becomes especially evident when you try a new project: for example an OSGI plug in project. You just run Maven with the proper archetype and bingo…

Secondly there is the repository system. You can configure a set of remote repositories, and just by specifying a library in your project file (.POM file) the maven build will try to fetch the library – of the version you specify too to your local machine, in a local repository – which is then shared amongst all of your projects. Again you notice how powerful this is when you download a maven-enabled project, and on the first build it goes and fetches all of its libraries which it depends on – unless you already have them locally. A large number of common shared libraries (e.g. most of the well-know apache projects) are available in the default repositories. It is possible to configure which external repositories are trusted and should be used.

The repository system has become effectively just another resource, to the extent that to install an OSGI bundle from the OSGI console (more on this next time) ‘mvn:’ is named as the protocol type for a given resource. The resource is then seamlessly retrieved; either from local storage, or from one of the configured remote repositories.

All clever stuff.

So from starting to look at Open Source middle-ware, I have fallen down a couple of rabbit holes. The maven excursion is definitely going to make me sit up and give that a much closer look (talk about late adopter!). The second rabbit hole for me was OSGI. more on that next time. Then it will be back on track for the open source middle-ware.

 

SOA is not just Web Services

This point has been explained in many other places, but still seems to need reiterating for some audiences:

SOA means Service Oriented Architecture

The most important word in that phrase could be ‘Architecture’, not the ‘Services Oriented’ piece – that just applies a modifier for the type of architecture; it does not mean that SOA is just the use of Web Services. Those are a technical interface mechanism and related standards that are indeed prevalent in any SOA environment, but that is only a part of it.

I, and my colleagues, often come up against a technical-biased viewpoint that implementing SOA is just a matter of using Web Services, SOAP, XML and WSDL for application integration. Maybe this comes about from the frequent over-use of the ‘Technical Architect’ job title, but that would be a different discussion with no real ending in sight.

In this context, an architecture implies a framework that is used within an organisation, whether it applies to business, information, application software or technology infrastructure. An architecture should include some form of blueprint and an associated set of principles to be applied to projects or solutions that fit within that architectural domain.

For SOA, a blueprint is an absolute prerequisite for any implementations to go into that environment (actually, for design and even project feasibility). Along with the blueprint should be a set of standards and principles that can be used to ensure the strategic SOA vision can be met. Tactical solutions may deviate from the strategic aims, but only if they can be shown to have strong justification for breaking the defined rules, and even then should have further plans to bring the solution into line for future strategic use.

The process of defining and applying SOA standards is that of governance, which is a form of quality control that applies to a greater level than is commonly seen in technical development projects where design and code reviews tend to take a project-specific view.

For SOA implementations to be successful, services should be designed to support business needs, so that they can be reused by multiple processes and applications, and using common approaches for interface definitions, data models, error handling and security. If these are not defined up front, as part of the architecture, it is highly likely that each set of services introduced into the SOA platform will be inconsistent, requiring additional effort for integration into any composite applications or services. Some of these points, especially that of security, can be very hard to ‘retro-fit’ if a standard is defined after the implementation of a number of service-based solutions. The same problems arise for any other standards that may be defined ‘after the fact’.

Hopefully these comments give a flavour of how SOA requires a broader level of thinking for application and solution design. There is a huge difference between a set of ad-hoc Web Services exposed in an ESB product and that of an architected design using SOA principles for compatible, consistent and reliable service-based developments.

If you are embarking on an SOA implementation, you really do need to form this holistic view of applications, services and business processes, with an architecture governance programme to ensure that the defined principles are applied. Of course, this is where the full ‘Enterprise Architecture’ approach comes in to play, with SOA being a subset of EA.

To those organisations, or more likely, development teams, that are just extending their EJB or COM components with a SOAP interface on an ESB, you should refrain from calling this approach a Service Oriented Architecture, as it most probably is not. It is quite likely to be just another bunch of incoherent application interfaces that will form a birds nest (or can of worms) with connections crossing all manner of boundaries. Well, that is my opinion of what I have seen happen, at least. Oh, one other whinge on common errors, it is ‘Oriented’, not ‘Orientated’. Shoot me if I use the wrong phrase!

SOA isn’t trivial, but if applied properly it can be a much better way of developing solutions within and across and organisation. So don’t be put off by the apparent overhead – doing SOA ‘properly’ is a good idea and should ideally be undertaken as part of a new approach to IT solutions that are using best practices in all possible areas.

I hope you now agree, that SOA is not just about exposing and using Web Services, and that if you are one of those committing the ‘crime’ of misusing or abusing the SOA moniker, please think again and take a step up to the greater view of architected solutions.

 

WebSphere Process Server v6.1 – new features

I’ve finally got round to reading this article on developerWorks. Things of interest that caught my eye:

• There are some changes to the BPC Explorer such as more meaningful column names etc – worth knowing about as it has user impact.
• Install time is supposed to be 1/3 what it was. Hurrah!
• Maximum business object sizes for various deployment platforms are stated in the document as follows – “In a 64-bit environment the maximum size is now 500MB, while 32-bit UNIX systems can handle 100MB objects. Windows has a limit of 50MB due to the smaller heap size.“
• Cyclic flow activity – the ability to loop back to previous points in a process was a real pain in the past on a project I worked on, and this new BPEL extension makes this easier. It is an extension though, so needs to be used with caution I guess.

In general the feel of it is that IBM are gradually adding some of the capabilities there are available in MQ Workflow and Interchange Server, but which were too low down the priority stack for inclusion in a previous release.

 

MQ and ESB Redbooks

Having recently done some work using WebSphere ESB and MQ, I came up with a list of IBM Redbooks that serve as useful references. Some of these are ‘must have’ documents for developers; others are good for additional background information. I’ll leave you to choose which.

All of these are available as electronic downloads (PDF files) from the IBM web site http://www.redbooks.ibm.com/ and are (mostly) identified with an eight-character id. I’ve included this id, plus a shortened form of the Redbook title, in the list below.

• sg246963: WebSphere Product Family Redbook
• sg247369: SOA Patterns, Design using WMB V6 and WESB
• redp4191: WAS V6.1 Technical Overview (redpaper)
• sg246451: WAS V6 System Management and Configuration
• sg247413: WESB and WPS Production Topologies
• sg247406: WESB SOA and SCA Connectivity
• sg247212: WESB V6 Getting Started
• redp4041: WPS and WID Technical Overview (redpaper)
• redp4304: WBI V6.0.2 Performance Tuning (redpaper)
• amqtac06: WMQ V6 for Windows, Quick Beginnings
• csqzaf09: WMQ V6 Clients
• amqzan09: WMQ V6 Using C++
• csqzaw12: WMQ V5 Using Java
• csqzal11: WMQ V6 Application Programming Guide
• csqzak10: WMQ V6 Application Programming Reference
• amqnar10: WMQ V6 Pub Sub User Guide
• amqzag09: WMQ V6 System Administration Guide

There are plenty more in addition to this list, but I found these most relevant for the piece of work I was involved with.

You may also find it useful to view the set of SOA Patterns as published by IBM. These naturally focus on the use of IBM products, but even if you aren’t using their platform, the overall patterns still make sense. For instance, you can substitute your own flavour of ESB or messaging technology, your choice of application server environment, plus your web technology.