IBM Worklight, making light-work of app development in the enterprise

Having recently spent time working on the IBM Worklight platform, I thought it would only be fair if I documented some of my findings. No disrespect to the IBM’ers, but its reasonably fair to say that documentation is a little sparse in places, so lets give a little back to the community by discussing some of the hurdles. Lets not dwell on what Worklight is, Andy has already covered this well in a previous post; but lets just dive right into some of the technical aspects.

General Thoughts

Development on the whole is a relatively straightforward process, even for someone like myself that often steers well clear of anything that involves web presentation technologies (it reminds me of dark nights at the university labs spending hours trying to get a button to align correctly, the night before coursework submission *shudder*).

The Worklight eclipse plugin provides a good drag & drop GUI builder, but with support only for dojo. I opted to drop dojo and go for jQuery. jQuery is very well documented, and is easy to get help should you require it. One of the main things I like about jQuery is its showcase and examples, they are documented very well and the learning curve is generally quite small, but also the themeroller, it becomes incredibly easy to customise the default colour scheme and drop the generated CSS into your app. It always amazes me how excited the marketing guys will get if you can add in the corporate colour scheme to your app (thanks Joseph!).

Continuous Integration

We’re big fans of CI here, so I was quite keen to understand how easy it would be to have our Worklight apps built from the command line, and ultimately on a Jenkins CI box. The chaps over at IBM have done a fantastic job of exposing an array of Ant tasks that help with building and deploying apps, you’ll almost certainly want to read through module 42 on the getting started page that covers these tasks:

• adapter-builder – Use this task to build your adapter and create the .adapter file
• adapter-deployer – Use this to deploy a .adapter file to a Worklight server (very  useful for deploying to a remote AWS instance)
• war-builder – Use this to build the server .war file that you will deploy to the application server (some manual tweaks are required)
• app-builder – Use this to build the .wlapp files that you will deploy into your Worklight container
• app-deployer – Use this to deploy your .wlapp files onto a Worklight server (useful again for remote deployments)

Lets have a closer look at each of those targets, and how we’re using them here at Smart421:

Getting the party started, with init

Firstly, grab the worklight ant jar (you’ll need to have purchased the WL Enterprise edition for this) and add it into your ant context like so :

<target name="init">
 <echo message="Loading ANT Tool"/>
 <taskdef resource="com/worklight/ant/defaults.properties">
 <classpath>
 <pathelement location="./build-config/worklight-ant.jar"/>
 </classpath>
 </taskdef>
 <property environment="env"/>
 </target>

Now you’re free to use the ant tasks anywhere in your build script.

Building & Deploying WL Adapters

You need to build each adapter individually, and then deploy each one. You can create the following ant targets to do that for you:

<target name="buildAdapters" depends="init">
 <echo message="Building all adapters"/>
 <adapter-builder
 folder="./adapters/TwitterAdapter"
 destinationfolder="./bin"/>
 <!-- Build your other adapters here, same as above-->
</target>

<target name="deployAdapters" depends="init">
 <property name="WLSERVERHOST" value="http://my_aws_ip_here:8080/SmartConf"/>
 <echo message="Deploying all adapters"/>
 <adapter-deployer
 worklightServerHost="${WLSERVERHOST}"
 deployable="./bin/TwitterAdapter.adapter"/>
 <!-- Deploy your other adapters here, same as above-->
</target>

Building the Server WAR

You can build the server war file using the war-builder task, as shown below. It is important to note however, that I needed to do some tweaking to the war file to avoid any post-installation configuration tasks. According to the Worklight forums, there doesn’t appear to be a way to include files in the WEB-INF when the war is created, which means that once you’ve expanded the war on the application server you’d need to manually replace the default web.xml and context.xml files (to set your datasources), this can be quite frustrating, so in true Blue Peter fashion, I’m updating the war file with files I created earlier.

<target name="warBuilder" depends="init">
 <echo message="Building the war file"/>
 <war-builder
 projectfolder="./"
 destinationfolder="./bin"
 warfile="./bin/SmartConf.war"
 classesFolder="./bin/classes"/>
</target>

<target name="updateWar">
 <echo message="Updating the war file"/>
 <war destfile="./bin/SmartConf.war" update="true" webxml="./build-config/web.xml">
 <metainf dir="./build-config" includes="context.xml"/>
 </war>
</target>

Building & Deploying the WL Apps

You’ll also want to automate the building and deploying of the wlapp files, you can do this with the following :

<target name="buildApps">
 <echo message="Building all WL Apps"/>
 <app-builder
 applicationFolder="./apps/Smartconf"
 nativeProjectPrefix="SmartConf"
 outputfolder="./bin"/>
</target>

<target name="deployApps">
 <property name="WLSERVERHOST" value="http://my_aws_ip_here:8080/SmartConf"/>
 <echo message="Deploying all WL Apps"/>
 <app-deployer
 worklightServerHost="${WLSERVERHOST}"
 deployable="./bin/SmartConf-all.wlapp"/>
</target>

Building the Native Application Distributable Binaries You’ve survived this far, and I’m thankful to you for that, however we’re not quite finished yet. Worklight will generate the native projects for you, but its your own responsibility to take those project directories and build the Android APK, or the iOS IPA etc. IBM will draw the line at this point, so you need to build them yourself, you can do this for all of the environments quite easily using additional ant tasks, android is the easiest :

<target name="client-android" depends="buildAndroid">
 <!-- Run the android native build, in its own directory -->
 <ant antfile="./apps/SmartConf/android/native/build.xml" target="release" useNativeBasedir="true"/>
 <!-- Copy up the apk into the bin area, for consistency -->
 <copy file="./apps/SmartConf/android/native/bin/SmartConf-release-unsigned.apk" tofile="./bin/SmartConfSmartConfAndroid.apk" overwrite="true"/>
 </target>

Building Blackberry and iOS apps from the command line is slightly more involved, and I feel they warrant their own blog post on that, alternatively, get in touch and we’d be glad to offer some assistance. Bear in mind you will need an Apple MAC to build iOS, for which we’ve installed a shared box in our build environment.

Other Gotchas

As with taking on board any emerging technology, there will always be plenty of head-scratching moments where the documentation is thin, and Uncle Google doesn’t provide much help, fortunately for you, we’re a nice bunch of guys here at Smart421 so we’ll share some of the things that had us pondering over a coffee:

• The trailing “/” in the Worklight server host URL is required, don’t ask why, it just is.
• The versioning conventions for Worklight are a little strange, 5.0.0.270 = v5.0 GA, but the developer edition is 5.0.2.407-developer-edition = 5.0.0.3.
• If you have an existing 5.0.0.2 WL server installation, don’t upgrade it to 5.0.0.3, it fails to upgrade all components and leaves you with some obscure error messages that are hard to trace. The best plan of action is to uninstall, install again, but make sure you check for updates at time of installing, via the wizard
• App crashes with Unreachable host? When you build and deploy the app to your device, it has the WL server IP hardcoded into it. The next day when you arrive at the office and hop onto the Wifi, DHCP gives you a different IP address…It’s a classic schoolboy error, but catches us out from time to time. A simple solution if you don’t have a spare box lying around is to install the Worklight server on AWS and deploy to the cloud, bearing in mind that it needs to be open to your mobile devices over the Internet in a real-life installation anyway.
• Results is undefined on adapter call. A subtle difference here, HTTP adapters use invocationResult.results, whereas SQL adapters use invocationResults.result. That last character makes all the difference.
• Response cannot be parsed, please contact support; this is an annoying error that you often see in the developer preview, just make sure you set the body onload to WL.Client.init() as mentioned here.
• Unable to use geolocation services on android? You’re probably seeing Caught security exception registering for location updates from the system, this should only happen in DumpRenderTree. Make sure you have the geolocations permission in your android manifest as detailed here.

Conclusion

On the whole, I was very impressed with Worklight, they are offering a lot of functionality over and above the standard Cordova project. Some of the errors I’ve encountered have been a little frustrating, as often my only source of help was the forums, but I can accept that it is a product in its early stages of adoption, and will probably go very far. I’m looking forward to working with it in the future.

If you’d like to have a look at some of the apps we’re creating, or generally just want a chat about Worklight and some of its capabilities, or Mobility in general, we’d love to hear from you.

 

WebSphere User Group (WUG) UK. September 2012

Flooded
Photo by East Coast Main Line

I have had the luck to attend the WUG at the Royal Society of Edinburgh on George Street again this year. This is a bi-annual event hosted in Edinburgh in the Autumn and, from this year, at IBM’s facilities on the Southbank in London in the Spring.

The good luck was attending, when maybe a third of people failed to go when the weather was bad, but the bad luck was when the East Coast mainline was flooded on the way down, causing a very late return home.

There were a few interesting sessions, including the Worklight acquisition for developing mobile applications on to a variety of target devices, including iOS and Android. Possibly more on that later. There was also a good session by Alan Chambers on sample use-cases for using WebSphere eXtreme Scale, which is a distributed in-memory caching technology. This is an interesting area, which merits further attention. The slide deck for the various sessions, including ones I could not get to, are on the WUG site.

David Sayers of MidVision also gave a talk about DevOps, which is the set of disciplines for bringing development and operations closer to each other. Although MidVision supply a tool in this space, David was keen to stay away for instances of tools, and to say that there is no magic bullet, and that it’s about process and people too.

A phrase which struck a chord with me went something like: “many firms don’t want to make a change in a production system because ‘Steve’ is on holiday and he’s the only person who understands this”.

It’s a spooky coincidence, as we have just published a development policy stating that all environments, and deployments to those environments should be 100% automated, as part of our policy refresh.

The presentation I want to elaborate on a bit this time, is the “How Lightweight is the [WebSphere] Liberty Profile” which is part of WebSphere Application Server (WAS) 8.5.

Simon Maple  (Twitter @sjmaple) – one of IBM’s technical evangelists on WAS – explained that this profile is an OSGi-based application server kernel which only loads up the libraries and subsystems, as you need them. The end result is a *very* lightweight application server.

So much so, that the session involved showing the application server running on a Raspberry Pi (£20-odd computer, the size of the palm of your hand, delivered as a circuit board).

To follow this up Simon then started up a WAS instance on his older android phone which was then serving up a blogging application via the phone’s wireless hotspot. I connected to it with my phone, and posted: “Amazing!” (yes Oscar Wilde won’t be looking over his shoulder), which then showed up on his monitor, along with some more imaginative posts.

I have the tooling, which was provided on a memory key in our “info” shared area for any Smarties to download.

The liberty profile tooling (eclipse plugins) even runs on a Mac, along with the dev runtime. Even though this runtime is not supported in production on Mac, this is a pretty major step for IBM. I would not have imagined it five years ago.
In terms of production use though, the liberty profile WAS is still a standard WAS install from the perspective of licensing… though I’m not sure how many PVU’s a Raspberry Pi has.

IBM also have a new Head of WebSphere Software, Stuart Hemsley, who was keen to get feedback from the delegates, both by announcement at the keynote, and by walking around during the breaks.

Our feedback was that the application server just costs too much compared to the likes of Tomcat and JBoss, and includes technologies which are starting to be less relevant (e.g. session clustering), as the application architecture moves to pursue session-statelessness. Yes you would expect to pay a premium for a big-league vendor-supported product, but not 10x as much.

It would be a shame for IBM to loose out on market share because of pricing, when they provide excellent tooling and support, as shown by a session on performance tuning the JVM… but that (as they say) is another story.

 

I think therefore I WUG

I also had the opportunity to attend the    Websphere User Group (WUG) meeting on 23rd March 2011 at Bedfont Lakes. The WUG is a very popular topic amongst colleagues at Smart421 as its a great community

As someone who doesn’t have much direct WebSphere experience on a day-to-day basis, I was wondering if I would struggle to follow the content. However, I can say that I was pleasantly surprised as the tracks were designed for a variety of skills levels. There were several talks that any Java developer would find interesting. There were actually 13 streams running over the course of the day. Many focused on specific IBM products (as you’d expect) but also some focused on more general topics such as Java, and OSGi.

The first session I attended was the WebSphere Foundation Update and Technical Direction in the WAS 1 stream. This session by Ian Robinson gave an overview of forthcoming features in WAS v8. While this was a very WAS specific session it also provided useful updates on several areas in the J2EE space. To download the slides, click here.

The second session I attended was in the WAS 2 stream on JAX-WS 2.2 and JAX-RS 1.1 support in WebSphere Application Server Version 8.0 Beta . The presenter, Katherine Sanders, a software engineer at IBM Hursley,  gave a very good introduction to these two technologies without being tempted to delve into a lot of overly-heavy WAS-specific details. To download the slides, click here.

The third session I attended was given by Simon Cashmore, a Lead Engineer within the Global Middleware team at Barclays Bank.  This talk, Changing the way Java Application Hosting is delivered at Barclays Bank , stood out by a mile as it was the only session in the Customer stream (c’mon WUG Committee, more like this please). It was informative because it focused on Barclays’ new approach to hosting Java applications. Barclays have essentially built their own collection of virtualised WAS instances that can be made available in days rather than weeks or months. Previously, projects would buy brand new hardware that was not shared or reused, so costs and timescales were sky high. Now they have a shared resource that can be used and reused much more efficiently – and more cost effectively. I’m sure Barclays shareholders will be very pleased to hear that  ;o)

The fourth and final session I attended was a talk in the Java stream on Generational Garbage Collection: Theory and Best Practices. This was focused on how the IBM JVM works, but Chris Bailey, a technical architect in the Java Technology Center (JTC) team at IBM Hursley, gave a very detailed description of it which applies to any JVM that implements Generational Garbage Collection. To get a copy of Chris’ slides, click here.

So if you’re in doubt whether you should attend a WUG meeting because you feel don’t have enough WebSphere experience then let me reassure you that any Java developer will find something of interest. There were also suggestions of adding a more business focused stream to future meetings to widen the potential audience even more.

Details of all WUG activities and events can be found here.

 

10 years on, WUG still has pulling power

Members of the WUG Board, past and present, cut the birthday cake. From left to right: Nigel Gale (founding Chairman), Simon Maple (IBM Representa tive), Alan Chambers (WUG founder and Board member), Chris Mason (Treasurer throughout the WUG's 10 years), and Jonathan Marshall (IBM Representa tive). Photo by kind permission of Alan Chambers.

On 23 March, over 200 members of the WebSphere User Group UK (WUG) and members of the WebSphere Integration User Group UK  descended on IBM Bedfont Lakes, Feltham, UK for the WUG’s spring-time gathering (2 annual meetings; March at Bedfont, September at Edinburgh). Smart421 was there with one or two of our bigs guns. More on that in a moment.

As longstanding members of the WUG, we get a lot out of these meetings - perhaps ’cos we also put  lot in. A significant number of our customer engagements require deep Java skills and several depend on WebSphere technologies in some way or another. Most speakers are IBM-ers, many out of Hursley, or sometimes further afield. Delegates from IBM, end-users of WebSphere and IBM business partners make up the remainder of the rich ‘ecosystem’ that is today’s WUG.

Smart421 Lead Consultant, Stu Smith, had his proposal selected by the Committee, which carried the catchy little title ‘Software Development Life-cycle with Message Broker in end-to-end SOA’ [Download the slides]. Nevertheless, Stu pulled a bigger crowd than usual with his piece and people seemed to appreciate his content and the very good Q&A session he triggered; for the last session of the day, it was a lively interactive exchange among attendees, who by then probably had their minds on the drinks reception or what they had to do to catch the early train home.

Alan Mangroo, one of our elite tekkies, attended for the educational tracks and was last seen diving in and out of sessions he has pre-selected. Knowing him, he’ll have made copious notes, so try to make a point of reading his separate blog [posted 08 April, click here].

The WUG has been running for ten years in the UK (yeah…I know !) and the Committee didn’t run past the opportunity to celebrate with drinks and two rather impressive cakes to mark the occasion. I’ve included a photo, courtesy of Alan Chambers, so you can share the moment with us. Proof –  if ever you needed it – that even tekkies have soul, so long as you bring the candles ;o)    Actually, I only remember cute miniature marzipan figures: developers with laptops.

As is often the case, Smart421 ran a on-stand prize draw for a bottle of Bollinger and appropriately Nigel Gale, the WUG’s first chairman (pictured, far left), was the one who swooped the 1st prize. Good timing I’d say. Hope you enjoy that, Nigel.

 

Software as a Service – Is it the right time to move services into or from the cloud?

One of the great benefits of being a member of the British Computer Society is being able to attend the very good conferences, forums and presentations they host and run, one of which I attended earlier this – “Software as a Service – Is it the right time to move services into or from the cloud?”.

The event was run by the CMA who are part of BCS from the BCS headquarters in London and was very well attended, with the likes of IBM, Shell, Sun, Fujitsu, Logica, Barron McCann, Experian, BT, Siemens, Ordnance Survey and many others sending delegates showing the rise in interest in Cloud Computing across all sectors. There were seven presentations in all, two of which were examples of companies that have embraced Cloud Computing, and a presentation from Rahoul Bhansali from Hudson & Yorke who has been heavily involved in the government’s G-Cloud initiative (which is something I am sure I / Smart421 will blog on later as the initiative matures). However for this blog entry I want to focus on the two presentations that caught my eye and overlay these with the two examples of companies embracing Cloud Computing.

The first presentation I want to focus on was from Nick Coleman of IBM (author of the Coleman report) who delivered a presentation on “The future directions of the Cloud Computing”. The presentation focussed on the message that “Cloud Computing is evolutionally and not revolutionary”, something which I wholeheartedly agree with; managed services have been around for years, virtualisation is not new and many companies have run / hosted applications on hardware hosted by a third party – Cloud Computing is about pulling these together and enabling different combinations of SaaS, PaaS and IaaS to be put together to meet the specific and individual needs of a company. Cloud Computing is transformative; it offers businesses, especially SMEs, a lot of benefits such as a reduction in costs, agility in deliveries, quicker time to market, easy to scale solutions etc, but there are still a number of considerations that need to be taken into account – security is still the main concern with the Cloud – in 2009 just under 50% of vulnerabilities found in web apps were not patchable, with Cloud based solutions you are not aware of what else is being hosted with your application, and where is your app being hosted? – these are all answerable but you will most likely have to dig down to get these answers and may even end up having to pay extra to resolve them. As Nick alluded to during his presentation, Cloud Computing is transformative, businesses and people need to adapt and be willing to relinquish control over some key aspects that have traditionally been kept in house. It was these considerations that featured heavily in the presentations from the companies that had embraced Cloud Computing – Ordnance Survey and EasyNet Connect.

Richard Britton who was up until recently IT Director for EasyNet Connect, presented on how they migrated their complex 50+ applications into a SalesForce Cloud based solution. They also surveyed their customers and:

• 73% of their customers said they would be using Cloud based solutions within 5 years
• 37% of their customers said they would be using Cloud based solutions within 1 years
• 66% of their large SME (50+ staff) customers are already using Cloud based solutions or plan to do so within a year

EasyNet have definitely embraced Cloud based solutions and plan to press forward with this as they grow.

Tim Martin and Ian Hoult from Ordnance Survey presented on how the OS OpenSpace product had been migrated to the Cloud; The OpenSpace application is used by many companies, including many local councils that seek the level of detail in the maps they provide. The application has between 5,00 and 10,000 updates on a daily basis that were shipped to all of the app consumers on a 6 weekly cycle, this equated to about 60GB worth of information so had to be loaded onto multiple DVD’s so was a very expensive process and a prime candidate to be converted to SaaS. In addition to the expense of shipping all of the updates, the demand for information on the local council web sites can flux quite a bit, the example they used was the recent increase in volume (around 80% above and beyond normal use) during the cold weather of people looking for salt bins. With this is in mind, it would have being very expensive for them to build the extra capacity that would be required to deal with a huge serge in requests. They decided to push OpenSpace into the Cloud to deal with these issues and can now serve 7 million tiles as opposed to 500,000 and for a third of the cost for delivering and now have a scalable and economical solution with reduced support costs – they can now add an additional server into an environment within 5 minutes if the need arises. With that said it wasn’t all plain sailing, they had to switch their database as the licensing with their existing DB was not “Cloud Friendly”, also the data has been split, so all personal data is held locally within Ordnance Survey and only the public data is hosted within the Cloud within the EU. All in all they have seen this as a huge success and will be delivering 2 more of their products into the Cloud, but will spend more time in looking into the contracts and licensing aspects which leads me nicely into the 2^nd presentation I wanted to focus on that was delivered by Richard Kemp from Kemp Little who specialise in IT litigation on the differences between cloud services and outsourcing.

Richard, who has worked with Google and SalesForce on this subject, was a very captivating speaker with an interesting subject, one of which I think is the most important for those looking at Cloud Computing. As I mention earlier, Cloud Computing is evolutionary and not revolutionary, we have all been involved in outsourcing in some way, shape or form in our careers and we all know that one of the hardest aspects is the contracts that need to be drawn up to support these. The traditional approach is a transfer out of assets and in return you receive a service contract. Richard said that with the ever increasing shift to Cloud based solutions, things are not changing as much as people think; you still need to set the KPIs, SLAs and service description but now certain items are becoming critical to these agreements such as:

• Performance – key service metrics need to be specified
• Availability – with the services being accessed over the internet who bears the risk of an outage?
• Data – Data Security is always a key item to any business, but when hosting that data in the Cloud you need to be sure that the data standards applied by the Cloud provider is that that you require; you should be able to access that data at any given point and most importantly, you need to be able to get your data back at any given point and in a usable format
• Exit – an exit strategy is key – being able to regain control of the service in a quick and timely manner

As well as these key points, there are some differences in the approaches taken to draw up the agreements for either outsourcing or Cloud:

• Outsourcing tends to be a custom / bespoke deal where as with a Cloud based deal being a one to many / generic deal
• With outsourcing the supplier is responsible for delivery; with Cloud based solutions the customer takes on the risk of their services being served over the internet and accept the risks with that
• Outsourcing generally work to a set up front free where as Cloud is generally pay as you use

I think the key thing is to do all of the hard thinking up front when thinking about moving to a Cloud based solution, look at the key points above, remember the criticality of your data and the fact that the customer is responsible for the data (make sure you specify where it needs to be hosted, what standards need to be adhered and make sure that the data can not be used with express consent), make sure an exit plan is agreed up front as this helps things nearer the end of any agreement and helps avoid litigation as it is always the lawyers who win!

All in all I thought the event was very good and gave some excellent insight as to what to bear in mind – in my honest opinion I think now is the time to be moving items into the Cloud, SOA is fast becoming adopted as much of a standard as OO and Cloud based solutions can be a truly cost effective, flexible and scalable solution for any business!!

 

More WebSphere Cloudburst

This week I attended a Cloudburst ‘Proof of Technology’ day at IBM’s Hursley Labs and I wanted to share my initial thoughts following this chance to play with the appliance. It was a scripted do this, then that set of labs along with some presentations so not necessarily a real tyre kicking but enough for things to click into place and some ideas around its value to form.

Colleagues have already blogged about this new appliance so I’ll keep it simple and liken it to a vending machine for virtualised WAS environments. You pick the one you want, press the button, wait and ta-da out it drops into your private cloud all ready to consume.

Its on-demand nature makes this compelling. Having spent significant periods of time creating virtual environments to play with various products this appeals to my fast food, consumer tendencies and although today it’s only WAS HV edition, DB2 HV is days away with Process and Portal Server HV editions following later.

Let’s imagine you are running WAS, have a number of servers that you are trying to maximise your investment in with a little virtualised sharing of hardware and you want clean, controlled dev and test environments to be simply and repeatably rolled out on demand and torn down when not required, with little fuss. You deploy the environment when you need to use it and when done tear it down. It doesn’t have to sit there consuming resources for any longer than the time you actually need it because when you do need it again you can faithfully recreate it. You can timebox the period that your environment is available and if this period is not extended, your environment is removed.

Thats what you get with this, and it’s so simple you can empower anyone who genuinely requires this level of access and automation to potentially remove a period of let’s be honest, faffing around to aquire the environment which could save huge amounts of time and money. There’s no elasticity, you pick your pattern and if you have 2 nodes that’s your lot. Cloudburst does do some monitoring but it’s not going to give you any more nodes when things are maxing out. It really is simple though.

It’s going to cost you, but nowhere near what you would spend scripting this level of automation and control and IBM are funding the ongoing development and maintenance costs with version 1.1 of the firmware coming so soon you can smell it and 2.0 well on the way.

Does it need to be a hardware appliance, I’m not sure but it is and its available today, in purple. Which must have upset the DataPower guys who already have a purple box in the form of the XB60 B2B appliance. So if you have both don’t rack them up too close together or you just know one day, someone is going to unplug the wrong one.

Is it possible to be brainwashed in a day? Possibly but I believe that if you have an investment in WAS you should have a look this appliance and what it brings to the table. Give us a call and we can put you in touch with a man who has one in a flight case with a pair of servers attached (he brings his own cloud!). He’ll come and see you, let you have a go and you can decide for yourself.

 

IBM’s smarter planet initiative

It’s been a full-on CTO day today, with a cloud computing partnership-related meeting this morning in London, then off to IBM’s Bedfont Lakes to meet Rashik Parmar, their European CTO, and now I’m waiting for the BCS Roger Needham lecture to start back in central London.

Rashik’s overview of IBM’s smarter planet initiative was fascinating – it was good to lift my head up from the day-to-day and consider some of the imminent and not-so imminent “mega-trends” in society and the IT world. Some of the key points that resonated with me were:

• In the industries in which Smart421 work, there’s been a lot of focus over the last few years in tuning the business processes to squeeze out all the low hanging fruit optimisations – and whilst there is always further tuning that can be done even in relatively stable industry sectors, the law of diminishing returns applies. Not that I feel that BPM has entirely lived up to the hype – but that’s another story…anyway – moving forward, business are going to need to apply business process reinvention (a term I just made up!) to innovate their way to additional markets and revenues.
• The transition to mobile devices to become the primary interaction mechanism of choice. Remember when it was said that iPhones were not a credible business tool? Didn’t take long for that view to be overturned did it…?
• The usual generation Y/collaboration argument
• An increase in instrumentation – measuring and monitoring things that we just didn’t or couldn’t before because the bandwidth, storage and compute capacity just wasn’t in place at the right price. As an aside, I worked out the annualised cost the other day of Amazon Web Service’s MySQL offering e.g. for their largest offering (basically a freakin’ beast…!), the “Quadruple Extra Large DB Instance” with a 64-bit platform, 68 GB of memory, 26 ECUs (8 virtual cores with 3.25 ECUs each – one ECU provides the equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor) – comes in at $3.10 per hour, which works out as $27,156 p.a., so maybe £20k p.a. or so. Seems like a steal to me. And they do all the DBA backup/patching admin for you/at your web service call request. As my colleagues have pointed out, there’s pros and cons of course and it’s not quite this simple so it’s not necessarily low risk enough for wide enterprise adoption yet.
• Things are moving so fast that it’s very hard to predict how things will develop – and so the best strategy for businesses is to back a number of innovation horses and accept that some will fall at fences along the way.

All these trends will drive a huge desire for more – more bandwidth, more compute power, more storage. And whilst there have been some very public hiccups along the way and there will be more, it’s an unstoppable train. Obviously vendors such as IBM are positioning themselves to be the providers of the infrastructure that provide the “more”. The only thing I can see that might slow down that train is climate change, and even then it is debatable.

 

Realities of WebSphere CloudBurst – part #2

Following on from the previous post, here are some more random notes about CloudBurst – pros, cons, features, limitations etc.

One thing that occurred to me was “Why is cloudburst a hardware appliance?” – it could just be a software appliance…well the key reason it seems to me is that it holds lots of OS/WAS admin passwords etc and so the ‘hardening’ of a hardware appliance with tamper-resistance etc is a key feature.

Deployment patterns and deploying

WAS hypervisor edition is actually an OVA image with 4 virtual disks, with multiple WAS profiles set up already – so the hypervisor edition is actually a VM image, rather than a customised WAS code base it seems.

There are patterns for a single server deployment, small cluster (3 VMs), large cluster (15 VMs) etc. You can modify a standard pre-packaged VM (e.g. add a fixpack etc) and then ‘capture’ back into CloudBurst as a standard catalogue VM for use in new patterns.

Control is available over whether certain pattern properties (passwords, memory size etc) can be overridden for each instance of that pattern or not.

A key point – keep track of any changes made to your VM patterns (e.g. any tuning done) and then ‘bake’ the in the pattern in CloudBurst so that any future deployments get the changes – otherwise they’ll be lost when you redeploy the pattern.

The first image transfer to each hypervisor can take up to 1 hour to transfer (obviously this is environment dependant)!

IP addresses are allocated on the fly when deploying, i.e. it pulls them from a pool of available IP addresses that the admin user sets up.

Script packages

Post deployment, CloudBurst can run any script, not just wsadmin scripts – essentially it ssh’s over to the VM and uses a zip file and an executable name (e.g. <some_path>/wsadmin.sh) with some arguments (e.g. what JACL file to run). ‘wsadmin’ scripts can be used against the deployment manager VM to install an application (EAR file) into the cloud cluster. Some “wsadmin” scripts are provided out of the box for common tasks – setting up global security etc.

Management facilities
CloudBurst provides a centralised access point to the consoles for each VM/WAS instance.

You can control separate permissions for each user (with LDAP integration), so you can have multiple users using a single CloudBurst box at one time, creating their own ‘private’ patterns etc.

You can use it to control the hypervisors to create snapshots of all the VMs in a particular deployment, so for example you can run some tests and then quickly recover the entire virtual deployment (i.e. all the VMs).

License management/metering etc – seems a pretty limited offering, it relies on admin REST APIs exposed by CloudBurst that are called by something like Tivoli etc.

CloudBurst admin console interface seems v..e..r..y.. slow to respond sometimes.

 

Realities of WebSphere CloudBurst

We’ve got a bit more under the skin of CloudBurst now and I wanted to post some info that’s not been written by anyone in Marketing…about the realities of the product (good or bad) rather than the saleman’s spin.

So what does it do? Well, in a nutshell it holds VMWare ESX and ESXi VM images of WebSphere Application Server (WAS) on disk and can install them to some IP addresses of your choice at your command and follows them up with some automated WAS admin scripting of your choosing. Some pre-created WAS Hypervisor Edition VM images exist (based upon Novell SLES 10 SP2) or you can create your own and package them, up using OVF (Open Virtualisation Format). There’s no runtime component to the product other than it’s VM management/deployment role, i.e. it relies on WAS XD if you want load balancing etc. There’s more to it than that but that’s the bare bones of it.

So what are the key use cases for CloudBurst – why would someone want one when they can install VM images themselves? Well, the key reason for use is to take the deployment cost out of an IT organisation. The creation of the OVF VM images is still going to be just as easy/traumatic as it was before, but once you’ve got a “pre-canned” environment setup in CloudBurst then you can roll that out repeatedly and with confidence with very little manpower required.

Who would use it? Well, if you get benefit from being able to ‘can a topology’ rather than just putting a single machine image ‘in a can’ then there could be real cost savings and agility/reproducibility benefits from being to roll out a defined topology to your private cloud very quickly and repeatedly. So if your organisation has many projects running throughout the year that need multiple dev, test, pre-prod, prod etc environments created and ripped down all the time, then you’d very quickly get a payback I suspect. It would also make you more likely to kill off an unused environment if you knew you could painlessly recreate it, allowing you to have less capacity needs overall.

The immaturity of the Open Virtualisation Format (OVF v1.0 was only released in March 2009) is a key constraint at the moment and this is an industry-wide issue – it’s early days. A key impact relating to CloudBurst is that each VM image is a bit of a beast at 20Gb minimum (not entirely clear why this is – maybe due to a WAS install being big anyway, due to the way virtual disks are managed in the OVF standard, and the images being uncompressed?). This directly impacts deployment times just due to the sheer volume of data to be shunted around, but it’s not immediately clear to me if this an OVF issue (it does have some compression support) or an issue with the implementation/use of the standard. If deployed more than once to the same hypervisor then deployment times can be accelerated as all this data doesn’t need to be provided a second time. It can take something like 7-15 minutes to deploy a VM image.

There are two key design approaches when creating your VM images (the best approach is probably a mixture of the two):

• use a ‘stock’ WAS image and then customise it with your coifing/settings and your EAR files installed etc, and create a new OVF image for deployment
• use a ‘stock’ WAS image and then do the customisation post-deployment using WAS admin scripting

So where’s it going from here…? Well support for Xen-based VM images must be likely as this is crucial for anyone who is going to ultimately deploy into Amazon EC2. Portal Server is already available on EC2 on a pay-as-you-go basis and WAS is coming. Also, it’ll be interesting to see if IBM support non-WAS images in later versions of the product.

 

WebSphere Integration User Group highlights

After a journey characterised by a conspiracy between parking meters and failed tube signals, I made it yesterday to IBM’s Hursley Park for the WebSphere Integration User Group meeting. Here’s the photo at a sunny Winchester station as proof…ok…so Jamie and I forgot to take any photos at the event.

The key note presentation was from Kevin Turner – ESB and Messaging Technical Strategy Manager. He covered IBM’s Smarter Planet initiative and then moved on to future architectural trends – the noteworthy points were:

• The 2nd wave of SOA is coming – there’s a significant number of ’1st wave’ adopters out there now who have been through the joy and the pain, and have now fully understood the upfront investment required and the potential benefits. This 2nd wave is likely to consist of organisations trying to join up their islands of SOA (probably built along departmental lines due to a sensible pragmatic strategy of ‘baby steps’ SOA), and so federation of multiple ESBs will be a key theme. Governance will be crucial here if these islands are ever to be consolidated into a virtual enterprise bus that solves the problems of visibility of services across ESBs, end-to-end security and manageability etc.
• Patterns – IBM are working on building some integration patterns support into their tooling (presumably WebSphere Business Modeller, WebSphere integration Developer etc) to allow an expert from an organisation’s ICC/ICoE to define the standard ‘pattern’ and therefore accelerate the development of instances of that integration pattern. The integration developer might just have to supply the pattern parameters for example, with many of the decisions such as how to manage errors etc already consistently solved for them.

There were a couple of presentations about specific MQ-related Supportpacs (MO71 and IH03) which I expected to be dull due to their nature, but the presenters managed to bring them alive – especially Paul Clarke. It was clear that he’d written and refined quite an impressive beast of an application over many years and was rightly proud of it.

Kevin mentioned CloudBurst during his keynote, and I managed to get some time with an IBMer later in a hastily arranged session to discuss it some more – I’ll post details in another blog post later as there’s quite a lot to report.

That means I missed the start of the next session about the developments in the WebSphere Process Server product to better support the human interaction aspects of BPEL processes (which have been sorely needed in our experience!). Paul Smith demoed the Business Space web app from WPS v6.2 which goes some of the way to addressing these shortcomings, with better human ‘override’ control of processes (skipping a step, repeating a step etc) and better visualisation tools for business users to use to understand where a particular process instance has got to, etc. This is clearly still a developing area of the product set though.

The last session I attended was a demo of rules from the recent iLog acquisition by IBM. An ex-iLogger Lenny Bromberg gave a very engaging demo which involved my colleague Jamie playing the role of “business user” to dynamically change rules that influenced the behaviour of a mock motor insurance quotation app. An interesting aspect of Lenny’s “pitch” was that essentially rules engines are 10 a penny, there’s several good open source ones out there if you want one, but what rules provides is a Business Rules Management System (BRMS), i.e. all the significant extras wrapped around the actual runtime rules execution environment that you need to make it really workable, manageable and governable. This includes rule development/editing environments (developer IDE and business facing), rule versioning, rule comparison tools, audit, simulation environments etc. Some other observations:

• Lenny’s experience from previous projects where they have integrated with BPM solutions (like WPS etc) is that they often find that the business process definition/BPEL has become ‘spaghetti’ as the business rules are not clearly separated from the business process – and so the use of an external rules engine enforces a good business process automation design practice, and leads to more maintainable BPEL etc.
• This is related to BRMS’s in general and not specifically iLog rules, but a weakness that I could see is that the rules rely on a good, stable business object model and we know from experience with numerous customers how difficult it is to get enterprise data models together and agreed. This is the potentially shifting sand that the rules are all built upon.

Many thanks to Mike Wyvill and Keith Guttridge and others for organising the event. Well worth the £28… )