“Work hard. Have fun. Make history.”

If that strapline doesn’t ring any bells, it’s because you may not have encountered an Amazon Web Services (AWS) employee yet.

At London ExCel next week, there will be no shortage of AWS cloud geeks ready at your service. You can’t miss them. They are distinctive – usually sporting their black polo shirts and cream chinos (standard issue AWS battledress, apparently).

But my guess is that for as many Amazonians that gather,  it will be nowhere near to the number of delegates flocking to Docklands. More UK business and IT people than ever are hungry to learn more about how they can use the AWS Cloud.

Yup, an AWS Summit is coming to town (29-30 April) and I have heard through the grapevine that online registrations have been buzzing with demand for weeks. No surprise there.

I have watched AWS Summits in the UK grow in popularity year on year. Now, it seems only a venue the size of ICC at ExCel is big enough. I think that probably signals something important about peoples’ appetite for new approaches as well as new technology.

At least that’s why I’ll be going. And fellow Smarties from our senior execs to our account managers will be too. We think there is an important conversation to be had with more large enterprises who, just like our existing customers using the AWS cloud, are looking for the greater business agility that the cloud offers.

For that reason, we will be making lots of our clever chaps available (stand S4) to give actionable advice and war stories rather than sales patter.  We’ll be showcasing several of our customer engagements, including:

  • Disaster Recovery in the AWS Cloud for Haven Power
  • Big Data analytics in the AWS Cloud for Aviva /
  • Service Transition to the AWS Cloud for ATOC
  • Super-fast migration to the AWS Cloud for Kuoni

Jeff Bezos, Amazon’s founder, has acquired a tidy fortune by anticipating the step change in online shopping, firstly with bookselling and then with online retail of … er, well…  just about everything. Those close to the action know that’s not the whole story, Bezos also foresaw the shift in compute and storage. AWS itself has grown rapidly to become the undisputed leader in the cloud computing space.

In amassing an enormous IT capability and renting it out to others on metered tariff basis, AWS effectively offers to everyone with a valid credit card a chunk of its own IT systems. This has opened up “on-demand delivery of IT resources via the Internet with pay-as-you-go pricing.”

Bezos likens the emergence of cloud computing to the change that happened years ago in the electricity supply industry. Instead of investing in their own individual power generation facilities, and paying to run it, businesses and consumers in increasing numbers opted to buy their electricity from operators running centralised power generators.

Visionaries such as Tesla, Edison and Insull knew the gains and the pains in the electricity market. Does Bezos know what he’s up against too? I’ve no doubt he does.

AWS is getting a helluva lot right. Innovation launches and price decreases have been relentless. Outstanding choice, reliability and security. If nothing else, AWS is a chip of the old block when it comes to Amazon’s now famous obsession with listening to what the customer wants, and getting it done.

Personally, I’m quite looking forward to seeing what AWS will be wheeling out next week. I not really a betting man, but a fiver says another price drop or another new whizz-bang offering.

Or maybe both.  What do you reckon?  Go on, leave a comment to let me know.


Please Rate and Like this blog. Share it using the social icons or via short URL 

Our readers want to know what YOU think so please add a Comment.


Fight in ice hockey 2009It’s been a pretty amazing 48 hours or so in the mega-cloud vendor space. We’ve rather lazily got used to continual price reductions from AWS, but this round of Google vs AWS price reductions are pretty special even given this context.

First Google announced some very big price reductions – it was the storage pricing that really grabbed my attention, at 2.6 cents/GB. But for the majority of workloads the compute costs are the dominant component, and so the 32% reduction in compute costs is probably more significant for many. It’s a minor point, but the Google announcement mentioned “reintroducing Moore’s Law to the cloud“, but Moore’s Law is of course finally running out of steam, e.g. according to Intel it’ll be game over by 2020.

AWS have responded with this, but interestingly seem to be calling time on the race to the bottom, knowing that they have a much more credible enterprise offering than Google I suspect. On S3 they’ve almost matched Google but not quite at 3 cents/GB reducing to 2.75 cents/GB with volume. Perhaps the bit that I’m most excited about is the price reduction of the M3 instance range by a whopping 38% (e.g. an m3.large in US-East is reducing from $0.225/hour to $0.140/hour), given that the M3 range is often our preferred weapon of choice these days. That’s a massive bang for your buck.

The next obvious thing to look out for is what Microsoft do with Azure pricing – the assumption is that they will match AWS as per their previous announcement to “peg” their pricing to AWS. Ouch – imagine being an exec and getting out of bed in the morning to find out that you need to drop your prices by 30-80% across the board!

[ADDED 2nd April - Microsoft have done just that - see announcement on their blog here]

So what conclusions can we draw from all this? Well here are mine:

  1. What’s cheapest today is not necessarily cheapest tomorrow – so optimise costs for the long term, not the short term. OK, if you just want some server or storage capacity for a short time then go with the cheapest I guess, but in reality I’m talking about enterprise workloads and it’s never “a short time” – storage is for life, not just for Christmas :), and the cost of moving between providers might outweigh any benefit. Also, the costs are now so low for some workloads (e.g. if I’m playing around with some feature on AWS) that they are trivial anyway – so convenience and sticking with whatever minimises any usage friction are paramount for me.So rather like when choosing a bank to save your money with, where you might want to go for the savings account with the best long term track record of consistent high interest rates rather than the headline grabbing “bonus” offer – when selecting an IaaS cloud provider it’s their trajectory that matters (and hence their ability to leverage mega-scale). It’s not a great time to be a sub-scale (and sub-scale these days still means freakin’ huge) cloud provider unless you’ve got some specific niche offering…
  2. In general, we don’t recommend buying AWS Reserved Instances (RIs) for longer than a 1 year term. The 3 year term often makes more financial sense at that moment in time, but in reality the prices are dropping quicker in a year than the additional saving. This makes sense really, as AWS virtually created the IaaS market only 8 years ago, so a 3 year commitment is still a lifetime in this market.In fact, now is a great time to buy 1 year AWS RIs as it’ll be a few months (you’d have thought!) until the next round of potential price drops – maybe timed to coincide with the next AWS Re:Invent conference in Las Vegas in November – so you’ll get the maximum saving. An exception to my point here is that sometimes 3 year RIs are useful for projects where the TCO needs to be completely fixed and predictable – i.e. cost predictability for a business case is the primary requirement.
  3. A mild concern about where all this is heading – in my view there’s enough competition in the market at present for it to be healthy (i.e. the consumer is winning at the moment), but there is a risk that all but the most massive cloud service providers are squeezed out and the resulting oligopoly means that prices start to creep up again. You could argue that Microsoft’s price pegging announcement is an early sign of an oligopoly forming – reminiscent of the behaviour of the supermarket sector in the UK (where 4 retailers have 76.5% of the market). We’re a few years away from this risk so I don’t think this should infuence enterprise’s investment and vendor selection decisions today.

We’re loving it – what a great time to be migrating customers’ IT workloads to a cheaper, more agile platform where the price is only going down!

CloudExpo2013 Our LocationWhat does last week’s Cloud Expo Europe tell us about the maturity of the market for cloud services in the UK? As an Amazon Web Services Premier Consulting Partner, Smart421 had a stand in the Amazon Web Services Village which gave us a great opportunity to have numerous customer conversations. Wayne Stallwood, one of the AWS Architects from our internal AWS Practice, supported our sales and marketing staff on the stand, and we compared notes afterwards to draw out the key themes.

First of all, one immediate observation was that people were more openly talking about hosting production/live systems in the cloud, i.e. not just the usual dev, DR and startups. We’ve been at this cloud game for about 4 years now and so it is far from new for us (although as a side note, it was interesting to hear Mark Russinovich from Microsoft Azure saying “the cloud is new” this week) and we started to see this shift at least a year ago if not longer. Some of the presentations at Cloud Expo Europe reflected that, for example with a talk about DDoS hardening etc – very much about live systems. There were lots of questions about performance stability, resources, scalability, reliability etc – again more enterprise-level considerations.

Smart421 stand

Smart421 stand

Balancing this though, it was somewhat alarming that some of the people coming to the stand still wanted to talk about the buzzwords without really knowing what they are…so we had a few openers where it was “so this “big data” thing…what does that do for me?” and if you looked at the name badges it was established enterprise people asking the questions. This tells us that there’s still a huge lump of “educational debt” to overcome in the enterprise space.

I had time to attend a couple of presentations but they were pretty awful – dull vendor pitches. You need to choose carefully at these events as the attendees typically don’t pay to attend, and so the bulk of the funding for the event has to be sourced from vendors, and hence they all get to present. There is always some great content though, you just need to be selective and accept you’ll get a few duds.

Instead, I devoted the bulk of my time to understanding Red Hat‘s direction of travel, especially in relation to OpenStack (as I’m fascinated by the cooperation and competition in this area, e.g. from Mirantis , and I’m interested to see how the delivery of private clouds plays out as enterprises use it as a not always sensible stepping stone to the inevitable destination of public cloud) – although inexplicably they were squirreled away on an upper floor and poorly represented in the online show mobile app and so pretty hard for people to find.  I also took some time to catch up with AWS colleagues old and new – including AWS Technical Evangelist Ian Massingham.

The Cloud Expo Europe event itself was co-located with Data Centre World (just over half the floor space) and Big Data Expo Europe (really just a thinly populated stream within the Cloud Expo event), and it was a bit odd to be wandering around the show floor and then stumble into the “dark side” with vendors trying to pitch cooling, racking and UPS power systems to me. I don’t want to build a data centre, ok, AWS has already taken care of that for me :).

The pure cloud content felt smaller to me that previous years, and so as a final thought – I wonder if this reflects not so much that the cloud market is going off the boil but more the opposite – that it’s now mainstream enough that it’s harder to raise so much interest for events that are riding the latest hype?

CloudBrokerBadgeThe first Cloud Expo event I attended 2 years ago was striking for the myriad of traditional hosting companies who were cloud-washing their offerings (usually quite blatantly and badly I felt).  Last year what struck me was the myriad of new small vendors selling niche cloud-related product offerings – data transfer optimisation, security products, management products, etc. I wonder what the theme will be this year? It’ll be interesting to see how many vendors are wearing the “I’m a cloud brokerage” badge at this week’s Cloud Expo.

Whilst I was at AWS’s Re:invent conference last November, one of the guest speakers at the partner day was Tiffani Bova, Distinguished Analyst from Gartner. Part of her presentation covered the topic of cloud brokerage, something Gartner have been talking about for quite a while, and something Tiffani and I had some debate about afterwards.

I must admit, it took me a while to wrap my head around the concept of cloud brokerage, partially as the pushing of the term was coming more from the analyst community than the rest of the cloud industry. Williams Fellows from 451 Research refers to this as “…a ‘cloudemic’ of firms now stamping ‘cloud broker’ across their service portfolios”. Tiffani’s view was that 90%+ of the AWS partners in the room (including Smart421) were brokers. It’s such a broad definition, e.g. Gartner’s definition is

Cloud services brokerage (CSB) is an IT role and business model in which a company or other entity adds value to one or more (public or private) cloud services on behalf of one or more consumers of that service via three primary roles including aggregation, integration and customization brokerage.

The great thing about definitions is that you can never have enough :). Way back in mid 2011 NIST published the following definition of a Cloud Broker…

NIST CloudBrokerageDefinition

The default view taken in society in that anyone with the title “agent” (aka broker) is looked down upon – e.g. estate agent, recruitment agent etc :). But by this definition I guess we’re all brokers in one way or another, even if it’s just combining toast and scrambled eggs to make the kid’s breakfast in the morning (aggregation).

Looking at what Smart421 delivers for our customers – we integrate different cloud and non-cloud services, we design and implement complex cloud environments and we add a 24×7 ITIL-based service management capability on top including ongoing capacity and cost optimisation. We also add value by handling access management, enhanced security, managing billing complexities and bringing new market innovations to our customers (as cloud service providers like AWS are still innovating and releasing functionality at an amazing rate, too fast for customers to keep up generally).  I guess that means I get to wear the badge too!

In a blog post tomorrow I’ll talk some more about one of the oft-touted cloud brokerage promises – that of dynamically migrating IT workloads across cloud providers (closely related to the dreaded phrase “bursting into the cloud”), and of making deployment-time decisions about which cloud service provider to use…and why I don’t believe in it.

PS If you happen to be attending Cloud Expo Europe this week, catch me on the Smart421 stand or let me know you are there via @SmartCTO

Getting ready for the 2014 conference season, it struck me that the technology revolution has moved from “E” to “D” – by that I mean Digital has replaced Electronic (as in E-Commerce) as the new “must have” conference title.

I remember back in the 1970’s when Digital was replacing Analogue – with the mass introduction of digital watches and calculators – so it amuses me to see it re-cycled. Obviously the generation adopting the word today don’t see the irony in it – nor do the companies inventing Digital divisions.

The serious point is that we’re struggling to articulate the impact of disruptive change on many axis simultaneously. Led by Cloud and Mobile and closely followed by Social Media, Big Data, the need for a secure on-line Identity and even ‘wearable’ technology (back to my old digital watch again) how do enterprises encapsulate the change ?

Cloud is probably the easiest to grasp as it is the most mature and is already releasing its technology potential, but now it needs to be recognised as commercial disruptor – it has already impacted traditional hosting vendors and subjugated “lock-in” contracts they felt were safe. Cloud provides not only a natural ‘leap frogging’ for new entrants into markets by reducing up front set up costs, but can also be a defensive strategy for those businesses trying to adapt to meet rapidly changing customer expectations and behaviours.

Mobile is a key driver changing behaviour – where the acceleration of mobile and tablet (e.g. non-PC) platform adoption is changing the location of the commercial interaction with customers. The mobility of smart phones and tablets has released the consumer from a seat at the desk (office or home PC) and consumers are “inviting” enterprises onto their commuter trains or into their sitting room as they ‘browse in the morning’ and ‘buy in the evening’.

Social Media has benefited from this informal interaction and given access to every review, post, tweet and blog – allowing research ahead of an eventual instore or online purchase – and making C2C communications the primary channel for feedback. Not only do we look up facts at the dinner table using our smart phones but we’re looking at everyone else’s opinion of that new camera, car or city-break as part of the selection process.

All that “opinion” needs a home and – adding it together with all the data produced from location tracking, monitoring and automated machine to machine communication – we have the exponential growth in the volume of data. Then you need tools and techniques to analyse that data (back to Cloud again).

Consumers are also demanding personal interaction which drives the need for Identity – allowing industries to start to drive up the quality and richness of exchanges to enhance customer experience.

So finally I come back to my original question – is my old Digital watch trendy again ?

Please Rate and Like this blog. Share it using the social icons below or via the short URL

Our readers want to know what YOU think, so please post a Comment.

Back in September last year I was one of the presenters at an event run by Qualitest where the focus was on big data – the possibilities, implications, testing ramifications etc.

I focused on how to make it real in your organisation and went through a Smart421 case study of the discovery and productionisation phases of a big data initiative, and the run cost implications when using an AWS Elastic MapReduce-based Hadoop infrastructure to run pig queries.  I’ve been meaning to share the YouTube content but Christmas got in the way :), so here it is…

As part of this event I gave a short 6 minute interview on the subject and my perspective on it…


…and here’s the presentation itself…

Vegasvenise1 Now that the razzmatazz of AWS re:Invent is over, I’ve taken to some time to step back from the specific new announcements and consider my higher-level conclusions, and what they tell us about the state of the cloud computing market in general.

As for the event itself, the whole “feel” was very start-up rather than enterprise focused, unlike the AWS Enterprise event Smart421 sponsored back in September. The main thing that struck me at the UK event was that wow – this is getting serious. I’ve been “banging on” for a while about the time now being right for enterprise adoption and the enterprise market reaching a tipping point – but the ability for AWS to draw a 500+ crowd from the UK enterprise market says it all, and also the fact that they felt confident enough to do so – even though there has already been a London-based AWS event earlier in the year. AWS subtly adjusted the tone and message for this audience and got it spot on I think – one of their challenges at other events like re:invent is generally that the audience is a mix of hardcore developers/Dev-op types, architects and enterprise attendees. This enterprise space is where Smart421 is used to operating and so it felt…well…just a bit more like home for us.

Hence I don’t interpret the start-up focus at re:Invent as a lack of credibility in the enterprise space really – it’s more that the new cloud-based startups are where the action and excitement is, and the innovation and crucially the adoption of innovation. I only attended one specifically “old school” vendor presentation at re:invent and compared with the other AWS-led sessions I attended – it was so D…U…L…L… that I just turned off. They demonstrated capability by telling – not by showing or doing.

As you’d expect, there was an army of AWS staff on site, but without exception I found them to be very well informed (there was no “give me your card as I’ll get back to you”…), very good, and uniformly excited about what they were doing.

In an least two presentations I attended there was reference to the flywheel effect whereby more and more momentum is created via a virtuous circle of innovation and customer adoption. Whilst I knew this before I went, I found it very striking that AWS’s success is founded on not out-doing the competition (at least not initially), but on out-innovating them. The crucial thing they have solved is how to innovate – sometimes in leaps, but usually in relatively small increments – but very fast, predictably and mercilessly. This creates some difficult tensions with their partners as mentioned in my previous post – ISV partners are especially at risk of being made redundant, but consulting partners like Smart421 have challenges too, e.g. every time RDS becomes more capable (multi-AZ etc) then it’s more likely to be used in one of our deployments, and needs less professional services to design, implement and support than just a database deployed on a virtual machine.

I can see how competitor X could maybe beat AWS today, e.g. maybe VMware can use their scale to defend their virtual desktop market against Amazon WorkSpaces – but who would take a bet against AWS not incrementally improving their offering faster than VMware? I wouldn’t. Innovation pace will always beat an existing market advantage eventually – it’s inevitable.

So does that mean AWS are unstoppable? Of course not – history suggests that they too will get blown away by another market disrupter in the future. Whilst we know this should be the case over a 100 year timeframe, at the here and now it seems impossible. But there are challenges – one challenge for AWS going forward was hinted at by Senior Vice President Andy Jassy – the person who has taken AWS from inception to where it is today – at the UK Enterprise summit. He came up with the quote of the day…

there’s no compression algorithm for experience

…which for me nicely sums up exactly how I feel about the services we offer to customers today. There’s an underlying tension in AWS’s market positioning in that on one hand they say that anyone can get started on AWS, just sign up and start your first instance etc – and it’s completely true – you can be up and running in minutes. On the other hand, enterprise customers know (or are starting to appreciate at least) that the real long term challenges are about managing change – change of architecture practices in their organisations, change of service management processes to consider capacity management differently, change in finance and procurement teams to get their heads around variable pricing etc. Also, once they get past the skunkworks stage, they quickly realise that this stuff is complicated. To get this point across, here’s all the separate Vision stencil icons representing various AWS services and features that we exploit to create AWS designs for our customers…and this was before all the new announcements at re:invent…

AWS Visio Stencils at Sept 2013

…and this set will clearly continue to grow further as AWS’s pace of innovation is simply exceptional. It’s a toolbox for our architects to pick from, so obviously we don’t use all the AWS services for every customer deployment…but you get the point I’m sure. Being intimately familiar with this ever-increasing set of capabilities and how best to exploit them is a full time occupation for our internal AWS Practice – and so I don’t realistically expect most enterprises to have the desire or ability to do so.

AWS will need to take care to balance their rate of innovation with the ability of the bulk of their customers to understand and adopt what they are offering. But that’s a challenge I’m sure their competitors would gladly take!

Share this blog by using the social buttons below or via the short URL

Please Rate and Like this post.  Our readers want to know what YOU think, so please leave a Comment.

Keynote room before we all troop in...

Keynote room before we all troop in…

Here at AWS re:Invent, Werner Vogels  (@Werner) was the keynote compere on day 3, and I quickly realised the formula. On day 2, Andy Jassy announced some new stuff with more of a business value message, but the real techy new announcements were for Werner to unveil. He also made light of the fact that all AWS ever do is bleedin’ mention Netflix, saying that a new drinking game had been created where you have to down a shot every time anyone mentions them.

Anyway, there was a lot of announcements…

  • PostgreSQL on RDS – this actually got a spontaneous whoop from the gathered audience, and so there is clearly a lot of pent-up demand for it. I note that it has multi-AZ support from launch, which hasn’t been the case for other databases on RDS.
  • A new I2 instance family – for uber-high IO
  • A new C3 instance family – for uber-high CPU
  • Global secondary indexes on DynamoDB
  • Cross region Redshift snapshot copies
  • Amazon Kinesis – stream processing that can handle huge data ingest rates and deliver it to a number of consuming applications or services.

So that’s enough for most vendors for a year of releases really.

Later in the day I attended the Amazon WorkSpaces session (which wasn’t previously on the re:invent agenda as it wasn’t announced!) to understand this new offering a bit better. I have to say that the first impression is one of… well… it’s a bit dull. What can I say – it’s a Windows desktop – even if you are accessing it via a laptop, an iPad or and Android tablet. It just does what it says on the tin. It’s the economics of it that are the really interesting thing. It’s not a market I know really well (i.e. the price points of the key vendors), but it’s fair to say that the Citrix guys on the expensive-looking stand in the vendor expo were not looking too happy – or very busy. In a kind of “my breakfast has just been eaten” way.

Interestingly I had a chat with an Amazonian on this point and his view was that Citrix could have easily launched a pay-as-you-go virtual desktop product like Amazon WorkSpaces if they had really wanted to, but I guess Citrix had the classic dilemma of whether to cannibalise their on-premise business model or not. The answer clearly is – you HAVE to do this, as someone else will regardless.

Both Citrix and VMware‘s share prices dipped following the announcement.

VMware’s response was as you would expect from a competitor – as detailed in Erik Frieberg’s blog post:

  • We welcome a new competitor in our market etc etc
  • Ours is better than theirs

What else could he say? But of course I strongly suspect he knows the real unspoken truth here – it’s not AWS’s first product launch you need to worry about (as it will always typically be relatively immature and lacking key features), it’s the pace of product updates that follow it that should concern you.

From a technical point of view, Amazon WorkSpaces uses the same protocol over the wire to “deliver the pixels” as VMware’s product – PCoIP, a UDP protocol that both parties have licensed. Only Windows 7 desktops are supported in this preview release, running on Windows Server 2008r2 under the covers. Other OSs are clearly in the pipeline, including support for browser based clients. Provisioning time is currently 15-20 minutes, but they plan to get that down to 5 mins when it comes out of preview. Active Directory integration is supported.

The day wrapped up with the re:Play party, sponsored by Intel. I had a chat with the Intel guy to understand what their motivation for such a strong sponsorship with AWS. Obviously AWS buy a shedload of Intel chips, but it’s more than that – I just think they know they need to be onboard with the way the wind is blowing, simple as that.

The party was good I must admit, although it’s never a good sign at a party when there is a queue for the men’s toilets but not the women’s :).  There was lots of entertainment laid on – 1980’s video games, Jenga, helicopter flying, laser-dodging etc – and then a set from deadmau5 – who I must confess I was not massively familiar with.  Good to feel your jeans shake with the sub-bass though….


I was off playing Centipede (badly) after 30 minutes of this.  I spoke to a Canadian (Dov Amihod, CTO from Lagoa – a really interesting start-up doing really high quality photo-real 3D image rendering for use in retailer catalogues etc as it’s cheaper than using getting a photo-shoot done) on the way back from the party and apparent deadmau5 is really big there, but he was a bit suspicious as to why he’s want to play in-front of 8000 geeks, for Intel :)

Experience deadmau5 by watching my short clip on YouTube here.

Please Rate and Like this blog. Our readers want to know YOUR opinion so please leave a Comment.

Yesterday at AWS re:Invent, Andy Jassy delivered the main keynote. As you can see from the photo below, the event was immense – the day before I was in the APN Summit so it was AWS partners only, and that felt big.

1384361167005But this was 9,000 attendees from 57 countries in a room. The photo doesn’t really capture the epic scale – which struck me as kinda like a metaphor for AWS itself, i.e. the scale of the administrative operation was off the chart, it was all very efficiently managed, and it gets bigger every year!

I thought it was interesting that they didn’t even “save up” the recent announcement about the 10% price reduction for M3 EC2 instances that was announced on 5th November for re:Invent. To me, this just shows how baked into the business model these regular price reductions have become.

In content terms, the three main new announcements were:

  • Amazon CloudTrail – the ability to log all AWS API calls to S3 for audit and compliance purposes. This is a nice feature that we’ve asked for before, but actually hasn’t been too much of a barrier to customer adoption previously, probably because we are typically managing the entire AWS layer for a customer anyway.
  • Amazon WorkSpaces – virtual desktops-as-a-service. Interestingly desktop “state” is maintained as you move between access devices, e.g. from laptop to tablet. We’re deployed virtual desktops in AWS for a number of customer projects – either desktops for key users in a Disaster Recovery scenario, or for developers who are located around the world and need a consistent desktop with known applications installed etc in order to access AWS-hosted dev and test environments. So I can see us using this new feature in future projects as I suspect the cost model in terms of the saved installation/build/ongoing patching effort of putting in a bunch of Windows Remote Desktop servers.
  • Amazon AppStream – HD quality video generation and streaming across multiple device types. This is related to another announcement that was made on 5th Nov – the new g2.2xlarge instance type, which has the GPU grunt to enables the creation of 3D applications that run in the cloud and deliver high performance 3D graphics to mobile devices, TVs etc.

Weirdly being at the event you get less time to look into these new product announcements and so you probably have less detail than if you were just reading about it on the web – after the keynote it was straight into a bunch of technical sessions.

I mainly focused on the data analytics sessions. First off, I got to hear about what NASA have been doing with data visualisation – I think all attendees expected to hear about exciting interstellar data visualisations, but it was actually about much more mundane visualisations of skills management, recruitment trends etc – and this in fact made it much more applicable to the audience’s typical use cases as well. There were some great takeaways about how to maximise your chance of success which I need to write-up at some point…

I then attended an excellent deep dive on Amazon Elastic MapReduce (EMR) – this covered Hadoop tuning and optimisation, architecture choices and how they impact costs, dynamically scaling clusters, when the use S3 and when to HDFS for storage, instance sizes to use and how to design the cluster size for a specific workload.

This was followed by some customer technical overviews of their use of RedShift. They had all migrated to RedShift from either a SQL or NoSQL architecture. For example, have deployed two RedShift clusters in order to isolate read from write workloads, but I felt that they had been forced to put considerable effort into building a proxy in front of RedShift to optimise performance – fundamentally as RedShift is limited to 15 concurrent queries and for their reporting workload, they are not in control of the peaks in their user’s demand for reports. So they’ve implemented their own query queuing and throttling mechanism, which sounds like a whole heap of technical and tricky non-differentiating work to me. A key takeaway from this session for me though was that the price-performance characteristic of RedShift had really worked for these customers, and given them the ability to scale at a cost that they just could not before. They were all achieving very high data ingress rates by batching up their data inserts and loading direct from S3.

The final session I attended was about a Mechanical Turk use case from InfoScout. Mechanical Turk is an intriguing service as it’s so different to the other AWS offerings – in fact it’s not a service at all really although it exposes a bunch of APIs – it’s a marketplace. Classic Mechanical Turk use cases include translation, transcription, sentiment analysis, search engine algorithm validation etc, but InfoScout’s need was for data cleaning and capture following an automated by fallible OCR process – capturing the data from pictures of shopping receipts taken on smart phones. The main takeaway for me was about how they manage quality control – i.e. how do you know and therefore tune and optimise the quality of the results you get from the workers executing your HITs? InfoScout use two quality control strategies:

  • Known answers – in a batch of receipt images that is handled by a Mechanical Turk worker, they inject a “known” receipt and compare the data captured with the known data on that receipt. This technique is good for clear yes/no quality checks, e.g. is this receipt from Walmart. This allows them to compute a metric for each worker as to how likely their other receipts have been accurately processed.
  • Plurality – send unprocessed receipt to more than one worker and see how consistent the returned results are. InfoScout use this to build a confidence score based upon this and other factors such as worker tenure etc.

The final event of the day was the re:invent pub crawl around 16 of the coolest bars in The Venetian and The Palazzo hotels. I’m guessing I don’t need to tell you much about that event, other than it started with sangria… :)

Tough but someone has to do it...

Tough, but someone has to do it…

Vegas baby!

Vegas baby!

I’ve survived my first full day in Vegas at AWS re:Invent, the annual Amazon Web Services shindig, although I must admit to being jet-lagged to hell. Handily nothing ever shuts down here so waking up at 2am is not a problem :)

The first day was dedicated to the AWS Partner Network (APN) Summit, and the #1 highlight had to be the announcement that Smart421 have been awarded Premier Consulting Partner status – one of only 3 partners in the EMEA region to be recognised in this way. This is the highest level that there is globally, and it makes me really proud of what our internal AWS Practice have achieved over our four year journey with AWS – this is not something that AWS give to any old partner! It’s recognition of the great customer case studies and deployments that we’ve jointly undertaken with AWS, and the investment in deep skills that we’ve made.

APNThe sheer scale of re:invent is pretty staggering. The venue (The Venetian) is enormous, the rooms are massive, and there’s a huge number of attendees with a very wide variety of interests – enterprise-level, gaming, HPC, start-ups etc. As I was at the APN Summit all day which was on its own floor, this didn’t really hit me until I went to the Expo part of the event at the end of day – where 180+ different vendors are touting their wares. It was a struggle even to walk through the room as it was so busy – although copious amounts of food and alcohol probably helped drive attendance :).

Here’s a couple of other takeaways from the APN Summit yesterday:

  • AWS have just updated the quote that they use to demonstrate the rate at which they are scaling their infrastructure. Anyone familiar with AWS will probably have heard before that one way of quantifying their rate of infrastructure growth is in terms of comparing with the number of servers etc needed to run the retail business at some point back in the past. Well – AWS has carried on growing, and so this comparison metric has had to be updated. They are now adding enough server capacity every day to power when it was a $7bn business – which is quite an incredible statement really. Cloud computing is indeed a scale game…
  • One of the big push areas from AWS is in driving use of AWS to host specific packages such as Microsoft Exchange, various Oracle business products (i.e. not just the technology components such as database, middleware etc), SAP, Microsoft SharePoint etc. Hence yesterday they announced some new partner competencies for some of these products. Personally I don’t quite get this – in my view, the cloud model is not so compelling for these kinds of IT workloads, as they tend to be very “steady state” in nature, not particular peaky workloads and if they are somewhat peaky, then you’ve usually got to have a resilient pair running all the time anyway and so they options for scaling down are limited. There’s a myriad of companies out there offering very price-competitive hosted Exchange and SharePoint models (like our sister company in fact) and they can exploit multi-tenancy across customers to drive a really low price point. Office365 (which wraps in Exchange and SharePoint with other stuff) is also the direction of travel for many enterprise customers. Having said all that, AWS are obviously seeing traction for these more enterprise-level software deployments otherwise they wouldn’t be aligning their partner model to it – as they are clearly not dummies given that they are giving the rest of the IaaS/PaaS market a bit of a hiding.

Today we have the opening keynote from Andy Jassy, and then we get into the more nitty-gritty technical sessions…


Get every new post delivered to your Inbox.

Join 1,122 other followers