Third Generation Outsourcing – SIAM

It was great to see National Rail Enquiries (NRE) win an award at the European Outsourcing Association Awards in Amsterdam last Friday (26 April).

In recognition of their SIAM outsourcing strategy (Service Integration and Management), NRE won the award for Best Multi-sourcing Project of the Year , beating strong category finalists 60k and Centrica (Centrica won this category in 2012).

Smart421 is pleased to be a large part of that initiative, performing the Managed Services element on top of an AWS Cloud platform for several key NRE applications.

As customers struggle with the chains of traditional SI relationships, Smart421 is providing agile delivery and innovation methods in the IaaS world.

Many analysts see this as “third generation outsourcing” and a change for good – and so do I.

 

Please rate and Like this blog.  If you can, please leave a Comment.

 

Adventures in AWS EC2 spot land

I was doing some Hadoop demo work last week for a customer and mainly just because I could, I used spot instances to host my Hadoop/pig cluster using AWS’s Elastic MapReduce (EMR) offering.  I thought I’d have a quick look at what the resulting costs were over the few hours I was using it.  I used a combination of small and large instances in the US-East region – m1.small for the master node and m1.large for the core nodes.  Note – these costs exclude the PaaS-cost uplift for using EMR (another 6 cents per hour for a large instance).

In summary – it’s dirt cheap….

What is more revealing is to look at this in terms of the % of the on-demand price that this represents…

So in summary, around an average saving of 90% on the on-demand price!  This is probably influenced by the fact that I was running the cluster mainly during the time when the US are offline.  We tend to get a bit fixated on the headline EC2 cost reductions that have frequently occurred over the last few years, and the general “race to the bottom” of on-demand instance pricing between AWS, Google, Microsoft etc.  Obviously not all workloads are suitable for spot pricing, but what I did here was deliberately bid high (at the on-demand price for each instance type in fact) knowing that this would mean that I was very unlikely to get booted off the instances as anyone bid higher if capacity got short.  As EC2 instance costs are so low anyway, we tend to not worry too much about optimising costs by using spot pricing for many non-business critical uses – which is a bit lazy really and we could all exploit this more.  Let’s do that!

 

If the world is going Big Data crazy then you need to know what it is, what it isn’t and what it means to your enterprise

The subcategory called Big Data is emerging out of the shadows and into the mainstream.

From left: Matt Wood, Chief Data Scientist at Amazon Web Services (AWS) with Robin Meehan, CTO at Smart421
Photo by Jim Templeton-Cross

What it is.

Definitions abound (who would have thought it? – quite usual in the technology market). For Big Data, we quite like the definition that originated with Doug Laney (@doug_laney), formerly META Group, now a Gartner analyst. It goes something like this:

 ” … increasing volume (amount of data), velocity (speed of data in and out), and variety (range of data types and sources)”

Gartner continue to use this “3Vs” model for describing Big Data.

Unsurprisingly, others are claiming Gartner’s construct for Big Data (see Doug’s blog post, 14 Jan 2012).

Still confused?

Put another way, Big Data is commonly understood to be:

“… a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools. The challenges include capture, curation, storage,search, sharing, analysis,and visualization. The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data, as compared to separate smaller sets with the same total amount of data, allowing correlations to be found to “spot business trends, determine quality of research, prevent diseases, link legal citations, combat crime, and determine real-time roadway traffic conditions.” read more on Wikipedia.

Big Data could be executed on-premise if you have sufficient compute and storage in your corporate data centre. And some do, especially some large banks, and with good success. Several solutions are already out there on the market;  Oracle’s Big Data Appliance is just one example.  But it does also beg the question “why would you” ?

If you don’t want the CapEx of purchasing more tin, or don’t want to gobble up capacity in your own data centre, then there are alternatives. For example, a cost model now exists with cloud-based compute and cloud-based storage (for example, think of Amazon’s announcement of 25 percent reductions in the price of Amazon S3, it’s storage solution) that puts Big Data in the Cloud well within the reach of all UK enterprises. A cost model like that islikely to win friends in procurement and in corporate governance as well as in IT.

Hinging on technologies including Apache Hadoop clusters, Amazon Elastic Map Reduce (Amazon EMR) and others, Big Data is delivering a degree of analytics and visualisation not previously possible at affordable levels.

Don’t just take our word for it, ask around. We could point you to other experts in Big Data, such Matt Wood ( @mza ), Chief Data Scientist at AWS.

What it isn’t.

Big Data isn’t business intelligence (BI). What I mean is that Big Data isn’t BI in any traditional sense of the term. It is altogether another level on from that. Granted that some tooling enterprises may own may be recycled for use in Big Data analytics. But it isn’t another species, it’s another race.

Big Data isn’t a lame attempt at reviving a management information system (MIS); those should be left to rest in peace.

What it means for you.

By now, if you’ve read this far, something should be niggling away at you that you could be missing a trick. I trust it won’t be those voices in your head again. But it might be your instincts telling you how Big Data could answer those tough business questions – y’know, those “I can’t be asked” questions that existing systems just cannot deliver.

Now, you would not necessarily get our CTO to come right out and say that Big Data is the next big thing. But evidence we are assembling so far does seem to point to a new capability to deliver. For those with an appetite to understand their business in new ways, Big Data is delivering tangible intelligence that lets them see new dimensions, new possibilities and new revenue streams.

I did get a full radar lock on something our CTO said in the summer. It was a throw away line at the time but it stuck with me and with others. So, when the time came to consider an appropriate go-to-market message for our quarter three (Q3) focus, we decided to wheel out his one-liner as part of our messaging.

“It’s not about survival of the fittest -
it’s about survival of the best informed”
Robin Meehan, CTO, Smart421 Ltd.

Making no apologies to Charles Darwin or evolutionists, the statement is resonating with decision makers in the enterprise space, not least those in the Insurance sector. Why?  Well, we think it is because a lot of the big insurers operate under many names in their brand portfolios.

The capability to see and understand impacts of brand activities, such as Insurance Quotes, delivered using Big Data analytics in the AWS Cloud, is illuminating new gains that would otherwise have remained out of reach.

Don’t forget – brand analysis is only one use case for Big Data in the Cloud.

If the world is going Big Data crazy then you need to know what it is, what it isn’t and what it means to your enterprise.

Agree?  Disagree?

UPDATE 05 Dec 2012 – Our economist friend Tim Harford  (@TimHarford) sent this hilarious tweet: The data do not lie. OR DO THEY? Muah huah huah! http://dlvr.it/2b2NS1

UPDATE 06 Dec 2012 – Robin and colleague Ben Baumguertel (@bbaumguertel) are attending the Big Data Analytics event in London today (organised by @WhitehallMedia ).

Please Rate this post and Like this post below. If you can, please leave a Comment.

 

Anti-IaaS and anti-AWS scare tactics

A colleague pointed me at an article in Computing the other day that starts off with “Retail giant Marks & Spencer is ditching Amazon as its online platform host“. As we are a leading AWS Solution Provider, interesting I thought, so I looked into it. The article itself it not misleading, but as usual the comments and interpretation it has generated have confused Amazon the retail store with Amazon Web Services (AWS) the IaaS/PaaS provider. For example someone made the comment “One is left wondering how many of AWS’s customers will move to rackspace or other companies of that ilk” – which is nonsense IMHO, and I’ll explain why.

M&S’s eCommerce offering is built using a white-labelled version of Amazon’s retail store – see this article in ComputerWeekly from 2007 when it was announced. I’m sure that some or all of M&S’s eCommerce site runs on AWS infrastructure, but that’s not the point really. I totally understand a retail organisation’s reluctance to use a retail competitor’s platform due to the reduced control over new functionality releases and the level of business data insight that this could give to a competitor. But those arguments do not apply to running a retail (with your own eCommerce software/platform) operation on AWS – if you follow good cloud architectural practices, e.g. encrypt data in transit and at rest, keep your encryption keys private and away from the cloud service provider etc.

In discussions with customers and vendors I see this confusion of Amazon.com and AWS all the time (sometimes deliberately to spread fear, and sometimes accidentally through lack of understanding). To be balanced here, the AWS offering has historically leaned on the Amazon.com brand and scale as a selling point, so I guess AWS need to keep making this distinction clear (which I’ve seen them do in every presentation). Anyway, either way it’s a red herring…and getting a bit tedious.

 

Smart421 achieves AWS Advanced Consulting Partner status

It is another exciting day for Smart421 with the news that we have been confirmed as the first UK enterprise to be confirmed on the Amazon Partner Network (APN) as an Advanced Consulting Partner by Amazon Web Services (AWS).

What does it really mean for Smart421 and our Customers?

In some ways, partner ‘labels’ are often seen as just that, labels, and can be given out like confetti. However, for Advanced Consulting Partner status, we had to put up some substantive evidence of various AWS capabilities including Customer references, minimum of $10,000 a month AWS billings, minimum of Business level AWS support (previously called Gold) and at least 10 trained AWS staff.

So before the cynics have a pop either at the program or at us, or both, I can reveal it does require proven AWS capabilities. That will sort the ‘wheat from the chaff’ so to speak

Does it change anything for us? – well yes and no…..

In terms of Operations, we already have a great relationship with AWS (since 2010 as an AWS Solution Provider and since 2012 also as an AWS Direct Connect Solution Provider) with access to a range of valuable contact points, from technical contacts through to Sales and up into Senior Management in the UK and USA. From my perspective as AWS Practice Manager, the existing relationship means regular face-to-face monthly meetings with our technical contacts in AWS, access to the product teams including ‘gurus’ based in Seattle, involvement in beta trials for new features etc. so the APN will just help reinforce those good relationships.

Perhaps the biggest change will be the impact of membership of the APN for our Customers. We are now able to be able to leverage a wealth of AWS resources on our Customers behalf. This translates into concrete deliverables as straightforward as documentation right through to support from technical architects during delivery engagements. It adds up to an improved level of confidence for our Customers that our proven AWS capabilities are fully backed by AWS and its rapidly growing global eco-system.

It means far more than a partner label to us and our Customers….

 

Impact of Amazon AWS IaaS* on Solution Architecture – bridging gaps between Application and Infrastructure Architecture

As Smart421’s partnership with Amazon Web Services (AWS) is slowly moving from pure infrastructure work into full cloud-based solution architecture design, I’ve recently spent some time analysing their IaaS platform from that angle.

My immediate goal with this was to better understand how the infrastructure side of my solutions are going to be affected by this paradigm shift moving forward. In other words, I was preparing myself to have a conversation with my infrastructure architect, let’s call him Jimmy, and not be caught showing my complete ignorance on the matter… again.

To my surprise I’ve found out that AWS, and IaaS in general, has a much more profound effect on my end to end solution design than just infrastructure, the key being how the “Servicelisation of Infrastructure” (sorry) provides a great mechanism to finally comprehensively address NFRs at the application view and close what I use to call “the leap of faith into the iron”… let me explain that.

We all know by now that our solution design is always going to start by a business architecture exercise that will feed the solution requirements, including functional and data requirements but also those dreaded non-functional requirements (NFRs)**, to our data and application architects, who in turn will produce the data model and component view, and that will be followed by our infrastructure architect, who will come up with the technical platform all those application components will run. Fine and dandy, but…

Dreaded NFRs? Why? Well, while it is quite straightforward for our data architect to create a logical data model out of the data requirements, or it is just BAU for our application architect to derive the system capabilities to cover the functional requirements, it is not easy for any of the two to cope with those NFRs… How does our data architect react to a NFR such as “the system shall process 2 million transactions per day”? Or what does our application architect think of the NFR “the site will provide the same response times independently of the location of the user”? Well, in most of the cases the reaction will be “Well, that’s for infrastructure to answer, isn’t it? Let’s pass those ugly things to Jimmy, he’ll know what to do with them…”.

So that leaves us with that situation we’ve been so many times, in which we provide a very detailed application architecture to Jimmy alongside a very long list of performance related requirements, hoping, and here’s the leap of faith I was referring to earlier, that our infrastructure hero will know how to put a lot of heavy equipment together that somehow magically will achieve those very ambitious performance goals…please raise your hands if that has never happened to you, or if that hasn’t inevitably ended up in all sort of performance issues detected just too late down the line, maybe a couple of weeks prior to go-live? Anyone?

Well, let’s see how Infrastructure as a Service may come to our rescue. Just do this simple exercise: go to the Amazon Web Services site and read all the documentation of the offering, all of it, trying to ignore the fact that it describes an infrastructure platform and rather set your mind-set as if you were looking at the functionalities of just another software system that needs to be part of your component architecture… Interestingly enough this made a “magic click” in my head, and suddenly I was thinking about my solution and my application architecture*** in terms of capabilities, functionalities and features that elegantly addressed all those long hated NFRs!

I’ve put this idea to test for a CMS solution I’ve been playing around with recently; I would have never typically defined capabilities such as a “Fast Access Data Access Layer” or a “Low Latency Distribution of Content” in my capabilities inventory, but suddenly my understanding of those AWS services such as ElastiCache or CloudFront made it dead simple to think about the NFRs and translate them to discrete solution components.

And what’s even more interesting is my design is not immediately coupled to the given IaaS platform as a result, not at all. As with the rest of the solution, these components and capabilities allow for a fit-gap exercise against the available options to be answered by my infrastructure architect: Do we achieve global performance by deploying the servers in our corporate data centres or maybe by deploying them in the AWS regions in the cloud? Or do I just keep my platform in a single location within my premises and use a CDN pull-zone for low latency delivery of static content? Quite a different proposition for Jimmy than the old “make it quick, boy”!

Now the problem is addressed where it should be in the design process, at the logical level, and decomposed into a set of features that achieve full traceability from the business into the application and then into the infrastructure layer and then back up again… the work of our infrastructure architect is now so much easier, as it is the predictability of our design exercise! Life is good!

Well just a thought in any case… I guess most of you were already through this learning, but just in case you had your own Jimmy’s suffering, this is a nice mental approach to try to bridge the gap.

* Infrastructure as a Service; I’ve focused this analysis to this type of cloud offering as it is probably where the biggest gap between architecture practices exist. In big organizations with dedicated software, middleware or platform architecture functions a similar situation will probably exist in which we could follow a similar approach with SaaS (eg. Microsoft Dynamics CRM Online) or PaaS (eg. Microsoft Azure)…

** It’s worth mentioning the usual problem of the business architecture exercise not really producing NFRs other than maybe a couple of fuzzy “it must be like really really fast” or “the site needs to look gorgeous”… this article is working on the bold assumption that our business analysts have been able to get blood out of stones and have coaxed the business into expressing real tangible NFRs to come along with the rest of requirements.

*** After all aren’t we solution architects just application architects with a good working knowledge of the other disciplines? At least I’ll confess that’s my case…

 

Microsoft Cloud Day

Organised by the UK Windows Azure User Group, this free all day conference provided a great opportunity to catch up on the latest developments, particularly given the Microsoft announcement a couple of weeks back.

Core to this announcement was Microsoft’s move into Infrastructure-As-A-Service (IaaS), and the key note by Scott Guthrie positioned IaaS (described as Virtual Machines) alongside Microsoft’s current Cloud offerings which to date has focused on Platform-As-A-Service (PaaS – now labelled Cloud Services by Microsoft) and Software-As-A-Service (SaaS – Office 365 for example).

Despite the lack of internet connectivity for a large part of the presentation (what is it with Cloud demos and loss of connectivity?!?) Scott did a great job talking through the slides, clearly describing the alignment of each of the deployment options: On-premise vs Virtual Machines vs Cloud Services vs SaaS.

In addition to Virtual Machines, the new Web Sites service was also discussed which gives Azure customers up to 10 web-sites and 1GB of storage for free (whilst in preview period, see here for further details). The demonstration showed how easy it is if you simply want to re-host an existing web-site on Azure whether it be ASP.NET, Node.js, PHP or even classic-ASP. So the new Web Site and Virtual Machine services provide a simple route to hosting applications on the Azure platform, but there is the added benefit of the Azure management aids, real time statistics and in the case of Web Sites incremental deployments and continuous integration (through TFS or GIT) too.

So where does this fit with Paas? Well Steve Plank from Microsoft provided some answers with another demonstration. With Cloud Services you get host of services to call upon including Storage, Database, Identity, Caching and Service Bus and the demo showed that if you design your application from the ground-up utilising these services, you benefit from an end-to-end application architecture that can be deployed and running in minutes at the click of a button. It is this architecture that really gives you the elasticity and flexibility in the places you need it.

A good day and exciting times with the options and landscape constantly changing. Nicely summed up by another Smartie (Andy Carter), ‘I guess there’s a load more stuff I need to learn about’, when a couple of days after passing the Azure certification MS announced the new services…(Well done btw!)

 

Smart421 a launch partner for AWS Direct Connect to the EU region

It’s an exciting day today as it has just been announced that Smart421 is an Amazon Web Services launch partner in the UK for the latest expansion of their Direct Connect offering. Up until today Direct Connect has only been available to certain regions in the US, but now UK customers can get the benefits of this improved connectivity as well. Smart421′s role as an AWS Solution Provider is to help customers reap those benefits by giving them a one-stop shop for full end-to-end connectivity from their premise(s) to the AWS EU region without relying on the Internet at all – except as a backup mechanism.

Our offering is actually a seamlessly managed combination of services from two parts of our parent company, the KCOM group. Smart421 provide an end-to-end experience for the customer and manage the provisioning of the AWS Direct Connect connection at Telecity Sovereign House in London, and we use our sister company Kcom to deploy and manage the “last mile” connection from Telecity to the customer’s premises. The immediate benefit to the customer is that they have one party responsible for a direct private connection from their premises to the cloud – not multiple suppliers to manage and triage etc.

I thought I’d just cover a few basic questions about Direct Connect that I’d expect to come up with customers…

Why would I want it?

The usual mechanism of accessing AWS is over the Internet, with user and/or administration traffic secured using a virtual private network (VPN). This gives privacy and authentication, but the network traffic is fundamentally still sharing your organisation’s Internet pipe and still going via the Internet along with everyone else’s traffic. Many of our customers have a default security policy that certain classes of network traffic must be deployed on a more private infrastructure, e.g. MPLS links etc – to give a greater degree of privacy, predictability and control, especially in terms of improved bandwidth, latency and availability.

Secondly, there is a perception issue with using the Internet – which often becomes more marked the further you move up the management chain . In fact, when talking to customers this is a classic objection that I sometimes hear – “we’re not comfortable using the cloud over the Internet”. Well now you’ve got a real choice – we can deploy an end-to-end private connection to AWS when required.

Also, you might be shifting significant volumes of data into and out of your cloud deployment, e.g. if you are performing big data processing using Hadoop/Elastic Map Reduce etc, or frequent data replication for disaster recovery purposes when you are using AWS as a logical extension to your on-premise data centres. In these circumstances, having greater control and certainty over the end-to-end connection between your premises and the AWS deployment is attractive.

What are the benefits?

In a nutshell, the key benefit is that your traffic is no longer subject to the unpredictability of the general Internet, and so basic metrics such as band with and latency will be far more predictable. For the connection from the customer’s premises to Telecity in London these metrics will be subject to strict quality of service guarantees, i.e. a bandwidth of X (you choose) with a defined maximum latency and an SLA (service level agreement) for the connection. For the second half of the connection from Telecity to the EU region in Dublin, you can expect superior network characteristics but there is not an SLA that defines guaranteed bandwidth etc. The initial adopters of Direct Connect in the EU region can expect an amazingly good network service given the price point – and our expectation is that over time AWS will have to introduce a degree of throttling/bandwidth management in order to maintain service levels…

What will it cost me?

…which brings me on to the costs. The bottom line is that Direct Connect is amazingly good value in our opinion. For a 1Gb/s port at Telecity it’s of the order of $216/month – i.e. virtually nothing. Unless you have your “on-premise” servers co-located at Telecity, then the costs for the “last mile” connection backed up by a strong SLA back to the customer premises will be much more significant. So guess what – you get what you pay for – no surprise there! For organisations relatively close to Telecity Sovereign House in network terms (e.g. in London) this makes Direct Connect a no-brainer really once your AWS usage becomes significant in terms of business criticality or data volumes, and it’s still highly attractive for an UK-based organisation.

Where might all this be going?

Finally – I just wanted to finish on why we think this is a really exciting development. For the EU region, this is the first step on the road for Smart421 to be able to offer a truly end-to-end service management offering – backed by strong SLAs for the end-to-end network connection and the AWS deployment itself. Over time we expect AWS to enhance Direct Connect with QoS (quality of service) guarantees, and we’re delighted to be in there at the start.

 

Clouds expected soon in City of London

City of London. No clouds (yet). Photo by iStock

Rumour has it that CIOs and CTOs from some of the UK’s leading organisations will be meeting together in the City of London next week for a specially convened roundtable focusing on their IT strategies.

Smart421 has reserved a major City of London venue to host this exclusive roundtable, designed to enable these leaders to learn from each other as well as to discuss their challenges with subject area experts on Cloud computing, including directors of AWS and our own CTO, Robin Meehan (who contributes regularly to this blog).

Senior executives share a common purpose – to drive their strategy forward and reinforce their leadership positions in their respective markets through embracing the new “art of the possible” where advances in technologies enable them to do so.

We will present that argument in the context of the specialised area of infrastructure as a service (IaaS).  IaaS is a layer of Cloud computing that seems far less sexy than the more visible SaaS front end. However, for those that possess the know how, it offers a set of enterprise-grade technical solutions that large business want to harness.

Expected discussion topics will include business continuity planning, disaster recovery (BCP/DR), “big data” analytics and service management in the Cloud.

You heard it first here…. (we’ll keep you posted).

 

New AWS Practice formalised same day as AWS pricing drops

Co-incidence ? – maybe. Or perhaps it shows again how the IT landscape is changing fast.

Today, Smart421′s CTO Robin Meehan [profile] announced the formalisation of a dedicated AWS Practice to exploit opportunities in enterprise-grade infrastructure-as-a-serice (IaaS). Steve Williams has been named as Head of the AWS Practice.

Smart421′s vendor neutrality is well known and this doesn’t mean we are compromising on that position. What it does mean is that Smart421 recognises that the market is voting with its feet and, in this space, the biggest player by far is Amazon Web Services (AWS).

Tweets since yesterday by Werner Vogels, AWS’s CTO [ @Werner] and Iain Gavin, AWS’s UK country manager [ @iaingavin], heralded a pricing change that from today, 01 July, allows business to benefit from a zero price tag on data transferred into AWS’s cloud – paying only for the hosting. This may seem subtle but in fact means a significant advantage in removing barriers that prevent some enterprises from making the leap towards AWS S3, for example.

Review the new pricing, click here.
Read AWS blog about the price drop, click here.

Feedback warmly welcomed, so please leave a Comment.