Third Generation Outsourcing – SIAM

It was great to see National Rail Enquiries (NRE) win an award at the European Outsourcing Association Awards in Amsterdam last Friday (26 April).

In recognition of their SIAM outsourcing strategy (Service Integration and Management), NRE won the award for Best Multi-sourcing Project of the Year , beating strong category finalists 60k and Centrica (Centrica won this category in 2012).

Smart421 is pleased to be a large part of that initiative, performing the Managed Services element on top of an AWS Cloud platform for several key NRE applications.

As customers struggle with the chains of traditional SI relationships, Smart421 is providing agile delivery and innovation methods in the IaaS world.

Many analysts see this as “third generation outsourcing” and a change for good – and so do I.

 

Please rate and Like this blog.  If you can, please leave a Comment.

 

SyncIpswich 2 brought us Continuous Delivery and Windows Azure Cloud

Wow! SyncIpswich’s second meetup and around 80 people crammed into the Eastern Enterprise Hub in the James Hehir Building at University Campus Suffolk. Many of the attendees were working for local behemoths like BT but there were also a good mix of bootstrappers, Start Ups and tech entrepreneurs with all kinds of backgrounds (even spotted a Chartered Accountant).

Organisers Carl Farmer (@CarlFarmer), supported by Anders Fisher (@atleastimtrying) and others have done a great job with SyncIpswich, which we are proud to sponsor. The focus of this meetup was on building software quickly with good practices as well as a nice introduction to the Windows Azure Cloud.

Talk no 1. Continuous Delivery

The first presentation by Chris O’Dell from 7digital (@ChrisAnnOdell) described how Agile practices (CI, Kanban, etc) combined with their architectural evolution to SOA have reduced code to deploy times to half a day  at 7digital.  And, by the sound of it, makes their developers more productive by getting away from “DLL Hell” that used to be the bane of any Microsoft Windows developer’s life towards a loosely-coupled set of services and a public API.

Chris raised some really interesting points around developing small fine-grained service components – not being that familiar with .Net myself this seemed to be similar to what we are doing in the Java world with OSGi and Service Component Architecture. I do like the policy of developing new features on the trunk (no feature branches) but making good use of feature flags rather than old-fashioned branch & merge.

They are also using Git for the code version control and Chris showed the inversion of the classic Unit Test, Acceptance Test, QA triangle. Some in our own organisation are raising question marks about the usefulness of very granular unit tests so the approach taken by 7digital of increasing the number of Unit tests is interesting.

There were a lot of questions from the floor, I was particularly interested in how the small kanban teams (about 6 or 7 members in 5 or 6 teams I think) interact when there are common services. This is a key problem that us SOA architects need to get right to get the best value on services. Feature Flags is something that we’ve also thought about in the context of simplifying application testing by, for example, switching off authentication for functional testing.

It’s great to see a company like 7digital competing successfully with iTunes and Amazon in the digital music space. I’ll be checking out their API (and their JLS back catalogue !) in more detail this weekend.

Talk no 2. Starting out in Azure

The second talk of the night was by Richard Astbury (@richorama) of Two10 Degrees ( @two10degrees). Richard gave a nice introduction to Cloud computing and in particular using Microsoft Windows Azure, showing a picture of a MS data centre under construction, which was something I haven’t ever seen before. I think it really brought home the sheer scale and commodity nature of the Cloud and these facilities being full of containers of kit that is just thrown away or recycled when it stops working.

Building a website on Windows Azure from scratch can use a few main pre-canned routes like the obvious “Website”, “Virtual Machine” and “Cloud Service”.

And it now includes a “Mobile Service” which is of particular interest to me. Sadly, I didn’t have time to chat to Richard about this but it’s on my “To Do” list to get a Hello Smartie mobile service up and running. In fairness, Richard did do two masterful demos for Website, including a node.js based site which he even launched from his home computer (a Raspberry Pi no less). As Carl tweeted:

Deploying to Azure from a remote RaspberryPi at home… Impressive stuff from @richorama !

— SyncIpswich (@SyncIpswich) April 25, 2013

Well done to the people of Ipswich for turning out and drinking all the sponsored free beer!
SyncIpswich will run and run.

Please Rate and Like this blog.  If you can, please leave a Comment.

 

Adventures in AWS EC2 spot land

I was doing some Hadoop demo work last week for a customer and mainly just because I could, I used spot instances to host my Hadoop/pig cluster using AWS’s Elastic MapReduce (EMR) offering.  I thought I’d have a quick look at what the resulting costs were over the few hours I was using it.  I used a combination of small and large instances in the US-East region – m1.small for the master node and m1.large for the core nodes.  Note – these costs exclude the PaaS-cost uplift for using EMR (another 6 cents per hour for a large instance).

In summary – it’s dirt cheap….

What is more revealing is to look at this in terms of the % of the on-demand price that this represents…

So in summary, around an average saving of 90% on the on-demand price!  This is probably influenced by the fact that I was running the cluster mainly during the time when the US are offline.  We tend to get a bit fixated on the headline EC2 cost reductions that have frequently occurred over the last few years, and the general “race to the bottom” of on-demand instance pricing between AWS, Google, Microsoft etc.  Obviously not all workloads are suitable for spot pricing, but what I did here was deliberately bid high (at the on-demand price for each instance type in fact) knowing that this would mean that I was very unlikely to get booted off the instances as anyone bid higher if capacity got short.  As EC2 instance costs are so low anyway, we tend to not worry too much about optimising costs by using spot pricing for many non-business critical uses – which is a bit lazy really and we could all exploit this more.  Let’s do that!

 

If the world is going Big Data crazy then you need to know what it is, what it isn’t and what it means to your enterprise

The subcategory called Big Data is emerging out of the shadows and into the mainstream.

From left: Matt Wood, Chief Data Scientist at Amazon Web Services (AWS) with Robin Meehan, CTO at Smart421
Photo by Jim Templeton-Cross

What it is.

Definitions abound (who would have thought it? – quite usual in the technology market). For Big Data, we quite like the definition that originated with Doug Laney (@doug_laney), formerly META Group, now a Gartner analyst. It goes something like this:

 ” … increasing volume (amount of data), velocity (speed of data in and out), and variety (range of data types and sources)”

Gartner continue to use this “3Vs” model for describing Big Data.

Unsurprisingly, others are claiming Gartner’s construct for Big Data (see Doug’s blog post, 14 Jan 2012).

Still confused?

Put another way, Big Data is commonly understood to be:

“… a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools. The challenges include capture, curation, storage,search, sharing, analysis,and visualization. The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data, as compared to separate smaller sets with the same total amount of data, allowing correlations to be found to “spot business trends, determine quality of research, prevent diseases, link legal citations, combat crime, and determine real-time roadway traffic conditions.” read more on Wikipedia.

Big Data could be executed on-premise if you have sufficient compute and storage in your corporate data centre. And some do, especially some large banks, and with good success. Several solutions are already out there on the market;  Oracle’s Big Data Appliance is just one example.  But it does also beg the question “why would you” ?

If you don’t want the CapEx of purchasing more tin, or don’t want to gobble up capacity in your own data centre, then there are alternatives. For example, a cost model now exists with cloud-based compute and cloud-based storage (for example, think of Amazon’s announcement of 25 percent reductions in the price of Amazon S3, it’s storage solution) that puts Big Data in the Cloud well within the reach of all UK enterprises. A cost model like that islikely to win friends in procurement and in corporate governance as well as in IT.

Hinging on technologies including Apache Hadoop clusters, Amazon Elastic Map Reduce (Amazon EMR) and others, Big Data is delivering a degree of analytics and visualisation not previously possible at affordable levels.

Don’t just take our word for it, ask around. We could point you to other experts in Big Data, such Matt Wood ( @mza ), Chief Data Scientist at AWS.

What it isn’t.

Big Data isn’t business intelligence (BI). What I mean is that Big Data isn’t BI in any traditional sense of the term. It is altogether another level on from that. Granted that some tooling enterprises may own may be recycled for use in Big Data analytics. But it isn’t another species, it’s another race.

Big Data isn’t a lame attempt at reviving a management information system (MIS); those should be left to rest in peace.

What it means for you.

By now, if you’ve read this far, something should be niggling away at you that you could be missing a trick. I trust it won’t be those voices in your head again. But it might be your instincts telling you how Big Data could answer those tough business questions – y’know, those “I can’t be asked” questions that existing systems just cannot deliver.

Now, you would not necessarily get our CTO to come right out and say that Big Data is the next big thing. But evidence we are assembling so far does seem to point to a new capability to deliver. For those with an appetite to understand their business in new ways, Big Data is delivering tangible intelligence that lets them see new dimensions, new possibilities and new revenue streams.

I did get a full radar lock on something our CTO said in the summer. It was a throw away line at the time but it stuck with me and with others. So, when the time came to consider an appropriate go-to-market message for our quarter three (Q3) focus, we decided to wheel out his one-liner as part of our messaging.

“It’s not about survival of the fittest -
it’s about survival of the best informed”
Robin Meehan, CTO, Smart421 Ltd.

Making no apologies to Charles Darwin or evolutionists, the statement is resonating with decision makers in the enterprise space, not least those in the Insurance sector. Why?  Well, we think it is because a lot of the big insurers operate under many names in their brand portfolios.

The capability to see and understand impacts of brand activities, such as Insurance Quotes, delivered using Big Data analytics in the AWS Cloud, is illuminating new gains that would otherwise have remained out of reach.

Don’t forget – brand analysis is only one use case for Big Data in the Cloud.

If the world is going Big Data crazy then you need to know what it is, what it isn’t and what it means to your enterprise.

Agree?  Disagree?

UPDATE 05 Dec 2012 – Our economist friend Tim Harford  (@TimHarford) sent this hilarious tweet: The data do not lie. OR DO THEY? Muah huah huah! http://dlvr.it/2b2NS1

UPDATE 06 Dec 2012 – Robin and colleague Ben Baumguertel (@bbaumguertel) are attending the Big Data Analytics event in London today (organised by @WhitehallMedia ).

Please Rate this post and Like this post below. If you can, please leave a Comment.

 

AWS Re:Invent unveils another yet another price reduction and DW-aaS

Unfortunately I didn’t manage to make a strong enough case to travel to Las Vegas in person , so I did the next best thing and watched the live media stream yesterday evening – it was just like being there, but without Tom Jones or any showgirls. The two big things from Andy Jassy (the AWS SVP) were an approx 24% storage (S3) price reduction across all regions from 1st Dec, and the launch of a limited beta version of datawarehousing-as-as-service. On the second of these, AWS Redshift (which is discussed in more detailed in Jeff Barr’s post here) is a direct challenge to the existing column-oriented database world, Teradata, IBM, Oracle etc. It looks really interesting and is a classic cloud use case and so it makes sense for AWS to tackle it – it requires large volumes of storage and compute power and is a traditionally high-CapEx market sector – I’m looking forward to playing with it..

As for the S3 price reduction…well, a 24% price reduction is a pretty amazing step change in pricing. In what other industries would have such dramatic changes in price? I wish it was happening to UK gas & electricity pricing . Having said that, Google storage costs currently start at $0.095 per GB per month, so it looks like AWS are price matching with Google. Microsoft Azure pricing was still at $0.125 per GB when I checked this morning, but presumably they will have to respond (to be precise – this is not quite an apples for apples comparison as Azure replication is over a significant distance whereas AWS S3 replication is between AZs which are separate but within some, but typically undisclosed, kilometres). As discussed before on the blog, I can’t see how the majority of smaller (and by that I still mean very big!) IaaS cloud players can possibly compete with this perfect storm of huge economies of scale and immensely deep pockets. Looking at our current AWS billing (which includes customer’s AWS accounts that we manage on their behalf), S3 storage costs only account for <5% of the total costs as the lion’s share of the cost relates to compute – so more price reductions here as well please!

[Update 30/11/12 - Since reading Jeff's post I've realised that these cost savings also apply to EBS Snapshots (der...of course you'd expect that), so this actually makes the cost saving from this one price reduction more significant, getting up to 8% or so]

 

Never mind the X factor – Cloud Computing showed it has the WOW factor at the UK IT Industry Awards

After all, there was really no need for our MD or our guests to become dispondent.

Having been named as a Finalist in the ‘Best Use of Cloud Services’ category, Smart421 may have been a tad surprised that the judges went for another Finalist at the UK IT Industry Awards.

In fact we were rather impressed with, and pleased for, The Met Office and PA Consulting who together saw off some pretty stiff competition to get the gong at the Awards Dinner in Battersea, London last Wednesday night (14 November).

And I was surprised that there actually wasn’t more gloating by the victors. You know how some marketing types can behave when tanked up on too much champers.

The UK IT Industry Awards did a great job to properly recognise the role of Cloud Computing for the first time when they selected the Weather Observations Website (WOW) which is currently available online as beta. See http://wow.metoffice.gov.uk/

And we think that this actually bodes well for the entire IT industry in the near and long term. It means that Cloud has proven that it has learned to crawl, walk and run. In other words, it means that Cloud Computing is winning its place as a resilient and credible alternative to on-premise-only lock in. And at an enterprise level, we think that must be a good thing.

As our customer Haven Power will tell you, being selected as Finalist at all was very encouraging. Their Disaster Recovery system architected by Smart421 and built in the AWS Cloud drew some significant attention from the press when it was first announced and, most likely, gave The Met Office a fair run for its money in front of the judging panel.

But putting the industry high jinks aside, the real winners are our customers who have fully functioning Cloud DR in place today as part of their business continuity planning strategy. Stuff like this is the sharp end of the Cloud revolution.

It has to be said, attendees were also impressed with our colleagues at Kcom (neighbour brand in the KCOM Group stable) who sponsored the on-site Twitter Wall. It turned out to be the unexpected big hit of the night and Kcom ranked top for tweets.

Never mind the X factor – Cloud Computing showed it has the WOW factor at the UK IT Industry Awards.

 

Three and Out ! – online banking apps, mobile networks and broadband outages point to why more resilience is now needed

“outages with online banking apps, mobile networks and broadband point to why more resilience is now needed” Martin Brazill

I’m beginning to feel like I’ve had a glimpse of the Stone Age. My personal BT Broadband was down for eight days which left a surprising large gap in my life.

Why ? Well, I needed to pay my credit card bill which I always do on-line in the evening – very convenient. So without broadband I decided to fall back to my O2 smart phone and my Nat West Banking app. Ah – only to hit problems with the bank, which was off line due to the widely publicised RBS problems. So I decided to phone up my branch – ah [again] no mobile network was available.

OK, so I survived a few hours “off the grid” but it did make me think that with our dependency on technology and communications we need stronger resilience. Up until now the pace of change has been so great that it’s been [comparatively] uneconomic to build resilience.

Now with the maturity of the Cloud solutions there are fewer costs barriers and no excuses not to build that resilience. Smart421 is already helping our customers with this – so if you want us to help you please form an orderly queue here.

And yes, I managed to get my credit card paid off too – eventually.  Phew.

 

Smart421 achieves AWS Advanced Consulting Partner status

It is another exciting day for Smart421 with the news that we have been confirmed as the first UK enterprise to be confirmed on the Amazon Partner Network (APN) as an Advanced Consulting Partner by Amazon Web Services (AWS).

What does it really mean for Smart421 and our Customers?

In some ways, partner ‘labels’ are often seen as just that, labels, and can be given out like confetti. However, for Advanced Consulting Partner status, we had to put up some substantive evidence of various AWS capabilities including Customer references, minimum of $10,000 a month AWS billings, minimum of Business level AWS support (previously called Gold) and at least 10 trained AWS staff.

So before the cynics have a pop either at the program or at us, or both, I can reveal it does require proven AWS capabilities. That will sort the ‘wheat from the chaff’ so to speak

Does it change anything for us? – well yes and no…..

In terms of Operations, we already have a great relationship with AWS (since 2010 as an AWS Solution Provider and since 2012 also as an AWS Direct Connect Solution Provider) with access to a range of valuable contact points, from technical contacts through to Sales and up into Senior Management in the UK and USA. From my perspective as AWS Practice Manager, the existing relationship means regular face-to-face monthly meetings with our technical contacts in AWS, access to the product teams including ‘gurus’ based in Seattle, involvement in beta trials for new features etc. so the APN will just help reinforce those good relationships.

Perhaps the biggest change will be the impact of membership of the APN for our Customers. We are now able to be able to leverage a wealth of AWS resources on our Customers behalf. This translates into concrete deliverables as straightforward as documentation right through to support from technical architects during delivery engagements. It adds up to an improved level of confidence for our Customers that our proven AWS capabilities are fully backed by AWS and its rapidly growing global eco-system.

It means far more than a partner label to us and our Customers….

 

UK Windows Azure User Group

Last Tuesday was another installment from the UK Windows Azure User Group (formerly London Windows Azure UG) which this time included a presentation from our very own Simon Hart, and as well as myself, two other Smarties also attended the event to heckle support Simon.

The session kicked off with an apology from Microsoft’s Mark Quirk (Azure Product Manager) due to last week’s Azure downtime. Mark described this as the first incident in two-half years and although any incident like this is never a good thing, it was great to see Mark there and helped reinforce to me just how important this stuff is to Microsoft.

The 1st agenda item was from Yossi Dahan (MS Technical Specialist) and given that he wasn’t feeling well he did a great presentation on Azure and how it can support mobile application development. Yossi demonstrated a mobile solution called BabelCam (source available here ) which goes something like this:

1. Someone takes a picture from a mobile device of a menu which is in a foreign language.
2. The solution then authenticates using the Azure Access Control Service (ACS).
3. Once authenticated, stores the image on Azure as a BLOB.
4. Passes the image to an OCR service to capture the text.
5. The text is then passed to Bing where it is translated into English.
6. Translated message is passed back to the mobile device.

A couple of things stuck out for me:

1. Due to the constraints of processing power on mobile devices, Azure enables developers to move as much logic and data as possible from the device up onto the cloud.
2. This helps deliver more powerful applications but also means that it is easier to support multiple device OS’s (Windows, iOS and Android) using the .NET SDKs.

The main realisation I had was how influential Cloud paired with Mobile will be and the combined potential that these technologies bring. Consumer-focused mobile applications (above Enterprise apps) require the type of scalability that only the cloud can provide. Couple this with the ease at which things like ACS can be configured to authenticate users via Live ID, Yahoo!, Google Mail and Facebook and MS are providing the building blocks for vastly scalable, quick-to-market solutions.

Following the beer and pizza (thanks sponsors), Simon Hart talked us through his soon to be published Azure reference architecture…in 45 minutes!

The solution enables you to notify your home-based media center of programmes you wish to record from any browser device. The effort and dedication shown by Simon in developing the reference architecture was obvious and the slides at the start helped frame the presentation; what initially looks to be a simple undertaking turns out to be quite a challenge. What Simon demonstrated was how all the facets of Azure can be used to overcome these challenges creating a solution that is:

• Secure (using ACS)
• Scalable (using the Azure Service Bus)
• Resilient (using an active-passive DR capability across regions)

It was great to see an example pulling all these different parts together in one solution…the difficult part, was grasping it all in the time! I think that Simon is penciled in for the next meeting (3rd April) to talk in more detail about the Service Bus, so it’ll be another late one for me but well worth it.

 

G-Cloud

Great news for Smart421 – we have been awarded a Framework Agreement on the new UK Government G-Cloud. This adds to our stock of real-world Cloud credentials – - but I was pondering what this means for us ?

Am I confident that the G-Cloud is going to change minds about the security of Cloud deployments – so opening the flood gates of opportunities?   No, of course not, but it’s certainly another step closer to making Cloud mainstream for public sector as well as private sector.

We’re probably not going to see adverts for the G-Cloud “CloudStore” on the TV for a while either.

But watch this space.