If the world is going Big Data crazy then you need to know what it is, what it isn’t and what it means to your enterprise

The subcategory called Big Data is emerging out of the shadows and into the mainstream.

From left: Matt Wood, Chief Data Scientist at Amazon Web Services (AWS) with Robin Meehan, CTO at Smart421
Photo by Jim Templeton-Cross

What it is.

Definitions abound (who would have thought it? – quite usual in the technology market). For Big Data, we quite like the definition that originated with Doug Laney (@doug_laney), formerly META Group, now a Gartner analyst. It goes something like this:

 ” … increasing volume (amount of data), velocity (speed of data in and out), and variety (range of data types and sources)”

Gartner continue to use this “3Vs” model for describing Big Data.

Unsurprisingly, others are claiming Gartner’s construct for Big Data (see Doug’s blog post, 14 Jan 2012).

Still confused?

Put another way, Big Data is commonly understood to be:

“… a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools. The challenges include capture, curation, storage,search, sharing, analysis,and visualization. The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data, as compared to separate smaller sets with the same total amount of data, allowing correlations to be found to “spot business trends, determine quality of research, prevent diseases, link legal citations, combat crime, and determine real-time roadway traffic conditions.” read more on Wikipedia.

Big Data could be executed on-premise if you have sufficient compute and storage in your corporate data centre. And some do, especially some large banks, and with good success. Several solutions are already out there on the market;  Oracle’s Big Data Appliance is just one example.  But it does also beg the question “why would you” ?

If you don’t want the CapEx of purchasing more tin, or don’t want to gobble up capacity in your own data centre, then there are alternatives. For example, a cost model now exists with cloud-based compute and cloud-based storage (for example, think of Amazon’s announcement of 25 percent reductions in the price of Amazon S3, it’s storage solution) that puts Big Data in the Cloud well within the reach of all UK enterprises. A cost model like that islikely to win friends in procurement and in corporate governance as well as in IT.

Hinging on technologies including Apache Hadoop clusters, Amazon Elastic Map Reduce (Amazon EMR) and others, Big Data is delivering a degree of analytics and visualisation not previously possible at affordable levels.

Don’t just take our word for it, ask around. We could point you to other experts in Big Data, such Matt Wood ( @mza ), Chief Data Scientist at AWS.

What it isn’t.

Big Data isn’t business intelligence (BI). What I mean is that Big Data isn’t BI in any traditional sense of the term. It is altogether another level on from that. Granted that some tooling enterprises may own may be recycled for use in Big Data analytics. But it isn’t another species, it’s another race.

Big Data isn’t a lame attempt at reviving a management information system (MIS); those should be left to rest in peace.

What it means for you.

By now, if you’ve read this far, something should be niggling away at you that you could be missing a trick. I trust it won’t be those voices in your head again. But it might be your instincts telling you how Big Data could answer those tough business questions – y’know, those “I can’t be asked” questions that existing systems just cannot deliver.

Now, you would not necessarily get our CTO to come right out and say that Big Data is the next big thing. But evidence we are assembling so far does seem to point to a new capability to deliver. For those with an appetite to understand their business in new ways, Big Data is delivering tangible intelligence that lets them see new dimensions, new possibilities and new revenue streams.

I did get a full radar lock on something our CTO said in the summer. It was a throw away line at the time but it stuck with me and with others. So, when the time came to consider an appropriate go-to-market message for our quarter three (Q3) focus, we decided to wheel out his one-liner as part of our messaging.

“It’s not about survival of the fittest -
it’s about survival of the best informed”
Robin Meehan, CTO, Smart421 Ltd.

Making no apologies to Charles Darwin or evolutionists, the statement is resonating with decision makers in the enterprise space, not least those in the Insurance sector. Why?  Well, we think it is because a lot of the big insurers operate under many names in their brand portfolios.

The capability to see and understand impacts of brand activities, such as Insurance Quotes, delivered using Big Data analytics in the AWS Cloud, is illuminating new gains that would otherwise have remained out of reach.

Don’t forget – brand analysis is only one use case for Big Data in the Cloud.

If the world is going Big Data crazy then you need to know what it is, what it isn’t and what it means to your enterprise.

Agree?  Disagree?

UPDATE 05 Dec 2012 – Our economist friend Tim Harford  (@TimHarford) sent this hilarious tweet: The data do not lie. OR DO THEY? Muah huah huah! http://dlvr.it/2b2NS1

UPDATE 06 Dec 2012 – Robin and colleague Ben Baumguertel (@bbaumguertel) are attending the Big Data Analytics event in London today (organised by @WhitehallMedia ).

Please Rate this post and Like this post below. If you can, please leave a Comment.

 

Microsoft Cloud Day

Organised by the UK Windows Azure User Group, this free all day conference provided a great opportunity to catch up on the latest developments, particularly given the Microsoft announcement a couple of weeks back.

Core to this announcement was Microsoft’s move into Infrastructure-As-A-Service (IaaS), and the key note by Scott Guthrie positioned IaaS (described as Virtual Machines) alongside Microsoft’s current Cloud offerings which to date has focused on Platform-As-A-Service (PaaS – now labelled Cloud Services by Microsoft) and Software-As-A-Service (SaaS – Office 365 for example).

Despite the lack of internet connectivity for a large part of the presentation (what is it with Cloud demos and loss of connectivity?!?) Scott did a great job talking through the slides, clearly describing the alignment of each of the deployment options: On-premise vs Virtual Machines vs Cloud Services vs SaaS.

In addition to Virtual Machines, the new Web Sites service was also discussed which gives Azure customers up to 10 web-sites and 1GB of storage for free (whilst in preview period, see here for further details). The demonstration showed how easy it is if you simply want to re-host an existing web-site on Azure whether it be ASP.NET, Node.js, PHP or even classic-ASP. So the new Web Site and Virtual Machine services provide a simple route to hosting applications on the Azure platform, but there is the added benefit of the Azure management aids, real time statistics and in the case of Web Sites incremental deployments and continuous integration (through TFS or GIT) too.

So where does this fit with Paas? Well Steve Plank from Microsoft provided some answers with another demonstration. With Cloud Services you get host of services to call upon including Storage, Database, Identity, Caching and Service Bus and the demo showed that if you design your application from the ground-up utilising these services, you benefit from an end-to-end application architecture that can be deployed and running in minutes at the click of a button. It is this architecture that really gives you the elasticity and flexibility in the places you need it.

A good day and exciting times with the options and landscape constantly changing. Nicely summed up by another Smartie (Andy Carter), ‘I guess there’s a load more stuff I need to learn about’, when a couple of days after passing the Azure certification MS announced the new services…(Well done btw!)

 

Clouds expected soon in City of London

City of London. No clouds (yet). Photo by iStock

Rumour has it that CIOs and CTOs from some of the UK’s leading organisations will be meeting together in the City of London next week for a specially convened roundtable focusing on their IT strategies.

Smart421 has reserved a major City of London venue to host this exclusive roundtable, designed to enable these leaders to learn from each other as well as to discuss their challenges with subject area experts on Cloud computing, including directors of AWS and our own CTO, Robin Meehan (who contributes regularly to this blog).

Senior executives share a common purpose – to drive their strategy forward and reinforce their leadership positions in their respective markets through embracing the new “art of the possible” where advances in technologies enable them to do so.

We will present that argument in the context of the specialised area of infrastructure as a service (IaaS).  IaaS is a layer of Cloud computing that seems far less sexy than the more visible SaaS front end. However, for those that possess the know how, it offers a set of enterprise-grade technical solutions that large business want to harness.

Expected discussion topics will include business continuity planning, disaster recovery (BCP/DR), “big data” analytics and service management in the Cloud.

You heard it first here…. (we’ll keep you posted).

 

It’s about the social enterprise, said Benioff

Marc Benioff, CEO of Salesforce with Michael Dell, CEO of Dell Computer

This week (14 Sept) saw an influx of over 6,000 people to London to attend Cloudforce conference at the Royal Festival Hall on the South Bank. It was the annual shin-dig for fans of Salesforce.com , the game-changing customer relationship management system hosted in the cloud.

Unlike at previous Cloudforce, it was clear on arrival that the Salesforce ecosystem had grown substantially. Elizabeth Decker, Director of Partner Marketing, had obviously been working overtime to assemble a noticeably bigger raft of partners, some specialising in apps development, some in implemention (among them some ex-Smarties). One observation was the very prominent role taken by Accenture (diamond sponsor of the event – which must have set them back a few quid); they featured a great case study for Schneider Electric in their 1.00pm breakout session on “The Cloud Enabled Business”.

But of course, everyone had come to see Salesforce CEO, Marc Benioff take to the stage in one of his now infamous, charismatic keynotes. Although I did overhear some grumbles that the content was a clone of his keynote at the recent Dreamforce event in San Francisco (itself something of a recordbreaker with over 45,000 attendees), he didn’t seem to disappoint his London audience.

Marc Benioff's vision of the Social Enterprise

Benioff’s new mantra was the “social enterprise“. Having launched “Chatter” just over a year ago, Benioff said he no longer speaks of Cloud 2.0. It seems he had a eureka moment somewhere after visiting people like Mark Zuckerberg (founder of Facebook) and others that the successfull enterprise is the connected enterprise.  From an article in Forbes magazine (Barrett, 2011)  it seems that Benioff instructed his half his development team to stop what they were doing and inject social media functionality inside Salesforce.

He asserted that a platform for social enterprise is truly transformational; Benioff was quick to remind the audience of the the role social media in changing political landscape; the “power to bring down dictators in countries and dictators in companies”, he squawked.

 ”It not about soft[ware] power or hard[ware] power but social power”.  Strong stuff indeed. Maybe you disagree?

At Cloudforce, Benioff shared the limelight with Michael Dell, CEO of Dell Computer (morning keynote and afternoon keynote interview, 14 Sept). They seemed very at ease and spontaneous. See my video clip here.

Marc Benioff at 2011 Cloudforce

Another guest was Angela Ahrendts, CEO Burberrys, the British fashion house, who was very vocal in her support for how Salesforce had helped her revolutionise her business whilst not sacrificing brand values.

The growth of Salesforce has been nothing short of staggering. And they’re not done yet. Acquistions such as Heroku and Radian6 are definitely technologies to watch.

Don’t forget to post a comment and I’ll let Marc know what you think…

If you weren’t there, or were there and want to relive the experience, then go online to the recorded video on the Salesforce website.

References

Barrett, V. (2011) ‘Mister Disrupter’ Forbes. 08 August. p.84.

 

Public cloud viability for mission-critical or business-critical needs

Cloud adoption patterns in the enterprise market (and the drivers behind it) is an interest area of mine, and an article I’ve written on this topic that has been published on the HPC In the Cloud website here.

 

In the end there is only (well…mainly) PaaS

Amazon Web Service’s announcement yesterday about their AWS Elastic Bean stalk offering was a significant step in their inexorable movement up the stack from IaaS towards PaaS (platform as a service), offering a Java PaaS based upon a Apache/Tomcat/Linux stack.

At the same time, the SaaS players are moving in the opposite direction, as demonstrated by Sales force.com’s recent acquisition of Heroku, the ruby PaaS provider.

Up until now, I’d have argued that PaaS has been the least mature of all the “aaS” siblings, with significant risk of lock-in, vendor failure risk etc, but this is rapidly changing and PaaS will become the main battleground between all the vendors as the SaaS players make their offerings more and more configurable/flexible and the IaaS vendors try and simplify their typically quite techy offerings for a wider audience.

There’s good further material of this topic on Krishnan Subramanian’s blog.

 

Cloud computing – now we know it is mainstream…

How can you tell that a new technology trend has hit the mainstream? Analyst reports? Marketing surveys? Conference attendance?

Well, sometimes the best way to get a real feel for how the economy is doing is to look at informal economic indicators such as how many skips there are in your street (as that means people are spending money on decorating or building work), or count the number of white vans going past your house in an hour.

Using this approach for cloud computing, I can now confidently state that use of the cloud is the norm, as my wife pointed out to me that there is an article about it in February 2011′s Ideal Home magazine. They list some of the main consumer-facing cloud offerings such as some SaaS email providers, streaming music (Spotify), Google Docs etc. I frequently use the flippant phrase “management by magazine” to describe how subjects enter the consciousness at the co layer of an organisation and become a priority for their direct reports, and now it is happening in a domestic setting as well!

I’ll have to order a back issue though as I must have missed Ideal Home’s 2008 article about service oriented architecture

 

Cloud dominates TechEd Europe

Firstly, apologies for this somewhat delayed post – put it down to the usual pressures of trying to get everything closed off before year end (and a little ‘help’ from my CIO ;o) )!

So my first visit to Berlin for Microsoft’s TechEd Europe event left me feeling drained attempting to get around as many bars breakout sessions as possible. The Cloud undoubtedly dominated proceedings and this is backed up by the fact that I only chose one Cloud specific breakout session, and yet out of the 20 or so sessions I attended I can hardly recall a session where the Cloud wasn’t mentioned! It highlights just how integral the Cloud is to Microsoft’s product strategy, covering 3 distinct service offerings; Infrastructure-As-A-Service, Platform-As-A-Service and Software-As-A-Service.

The keynote (watch it here) provided a clear steer on how Microsoft see the end-to-end application landscape evolving; user-centric portable devices, hooking into Visual Studio (VS) applications/services hosted in the Cloud. On the portable devices side the recent release of Windows 7 Mobile and the ease at which applications can be written in VS2010, provided an insight into how Microsoft see mobile apps becoming pivotal for businesses (anyone can get started using the free tools). Not much to shout about here in hardware terms but development wise there appears to be great potential for developing rich business applications for mobile devices.

Software-As-A-Service was headed by Office 365 (Microsoft’s answer to GoogleApps) and provides slick browser based versions of your usual favourites; Word, Excel and PowerPoint. What’s impressive here is that you also get Exchange Online, SharePoint Online and Lync Online (MS unified communications suite encompassing IM, Voice, Email and Web Conferencing). With pricing options for both SMEs and Enterprises, organisations have the opportunity to utilise some great tools with low up-front costs, that will help streamline communication and simplify company collaboration (both internal and external via federated Lync environments).

Infrastructure-As-A-Service facilitated by Hyper-V Cloud technology provides businesses with a self-service, elastic infrastructure platform. It gives you the opportunity to utilise vast amounts of processing power without the prohibitive expenditure you’d usually face when procuring your own tin. I was impressed with the Service Templates that simplify deployment and management of applications spanning multiple tiers. Remember however, that you’re still responsible for the patching, upgrading and management of all the servers deployed, but you get the commodity based benefits the Cloud can offer. Interestingly, Microsoft mentioned that financial assistance was available to companies wishing to be guinea pigs on this new offering.

Platform-As-A-Service is obviously where Windows Azure comes into play, providing a robust and scalable platform to host business applications. This was described as the panacea and it was clear to see alignment with all development technologies attempting to abstract the solution from where it may ultimately be deployed. This was demonstrated with VS2010 deployments, particularly when using AppFabric but also SQL Server in the form of Data Tier Application Development. The development tools are evolving to make it as easy as possible to switch between Cloud and non-Cloud based deployments, and clearly Microsoft want to remove as many obstacles as possible so that choosing Cloud services is literally one click away.

The other item I want to highlight was the effectiveness of Twitter at an event like this. Not only was it useful for me to keep up with the latest celebrity gossip, but I was really impressed with the ability for Microsoft to get immediate feedback on the event and breakout sessions. I can see it providing businesses with a great medium for assessing the impact of marketing campaigns etc. but also data-mining potential discovering patterns in comments. On this theme I recommend you take a look at this. A great week and highly recommended.

 

Salesforce.com – why has it been the biggest SaaS success story?

Salesforce.com are generally always quoted as the definitive success story in the SaaS space. They’ve grown their offering over the years, starting with sales force automation, moving into broader CRM, contact centre management, PaaS with Force.com, collaboration with features such as Chatter and now Java application hosting via VMForce.

Their revenue growth is quite amazing, as shown by the following graph. In terms of pure revenue, they are massive when compared with some of the other leading SaaS vendors.

* Note: Some of the 2010 data is based on trailing twelve months

What is even more interesting though is to look at the company revenue growth with respect to a common date in the past, e.g. 2006.

Here you can see that SuccessFactors has actually growing faster than Salesforce.com over this period, and NetSuite has tailed off somewhat (although it is still growing at say circa 10% per annum, a very healthy rate that many companies would kill for – it is just the growth rates of the other two vendors that make it look a bit ordinary!).

Of course this data needs to be reviewed with some caution. Salesforce.com are much bigger and therefore have a diminishing market to try and capture, whereas the other players are smaller fish in their respective ponds. It certainly looks like Salesforce.com got off to cracking head start though! Also – it’s not all about revenue – I could have looked at number of subscribers, or profit etc. Vendors have been in a “land grab” phase and so will have been sacrificing revenue to some extent and profit for sure in order to gain market share, and to maximise longer term revenue opportunities and market capitalization.

So why is Salesforce.com adoption faster than that for other SaaS vendors? Well, for sure their execution over the years has been excellent – they’ve obviously got the pricing right and kept it right, and carried on adding functionality despite starting from a very low base. Whilst SaaS vendors are generally thought of as young upstarts, Salesforce.com have been around since March 1999, and so they’ve had a long time to continually add to and refine their product offering.

But I would suggest another reason other than just excellent execution of their proposition in the market – I think they picked one of the best business process to address first – sales. Why? Well…

• Sales staff just don’t like keying in data about their opportunities, prospects etc – they’d rather be out on the road selling than doing “admin”. This was recently described to me as “attention deficit disorder”…
• The basics of the sales business process are relatively straightforward and the same across different vertical markets, i.e. capture prospects, generate opportunities, convert to business.
• The sales process has relatively few immediate integration needs – it makes sense “on its own” without integrating it with other processes/systems, at least in the early days of usage

SuccessFactors chose Human Capital Management (HCM) as the business processes to which they would apply the SaaS model. Again, I think HCM is a good process to select for a SaaS startup, as the business process can function in a relatively standalone manner, e.g. you could live without integration with payroll, etc in the early days of a “trial” deployment.

But what about NetSuite? They selected Enterprise Resource Planning (ERP)/Financials as their target SaaS-ified business process. ERP processes are typically quite likely to require tailoring to specific vertical markets, and both ERP and Finance processes are also likely to need integration with existing customer systems as a “day one” deployment requirement.

So, the lesson to learn from this is that whilst execution is all important in the delivery of a SaaS offering, the fundamental business processes that a vendor decides to target are a significant deciding factor that influences the size of market you can address, how successful you can be in it, and how quickly.

 

Cloud impacts on Enterprise Architecture presentation at Ovum strategy day

A couple of days ago I was one of the presenters at an Ovum/Butler strategy briefing at the Ambassadors Hotel in London. Smart421 were sponsoring the event and I wanted to share something real relating to cloud computing – and not just give a thinly-veiled sales pitch (not really my style – and also I’ve sat through enough of those from vendors/SIs to last me a lifetime) or explain what IaaS/PaaS/SaaS are to an audience for the millionth time.

So, after some steering from the Ovum analyst who was performing the introduction to the subject, Laurent Lachal, I settled upon the subject of what organisations need to do and consider today in their Enterprise Architecture teams in order to gear up and exploit the opportunities provided by cloud computing. It’s always better to have a map before you set off on a journey, and so I explained how we would use the Open Group’s TOGAF architecture development method (ADM) to help us navigate and address all the impacts of cloud computing in an organisation. Anyway, the presentation all went well with just the right amount of pre-match nerves to get me excited before I went on – and some good audience questions. Although my colleagues tell me that apparently I have a couple of bad habits when presenting that I need to work on . Drop us a line if you would like a copy of the presentation.

The main observations that I wanted to share in this blog post were relating to the nature of the audience, as they provided an interesting insight into the cloud computing market itself. As this audience had been generated from a list Ovum subscribers mainly, they were quite a different mix to other cloud events I’ve been to or presented at:

• Awareness – The level of knowledge of the subject was relatively low. You might say “that’s why they’ve come to a cloud briefing session you dummy” and that would be fair, but I am contrasting this to the knowledge levels I’ve seen at other, perhaps more techy events. My conclusion – it is very easy to get misled about how well understood the subject is by the market when you work in a “consulting bubble”. There is still a significant amount of “educating the market” to be done.
• Strategy – When the audience were asked “who has a cloud strategy/adoption plan in place today?” that the answer was – virtually nobody out of 50+ delegates. One hand went up.
• Execution – What was clear to me from the various vendors/presenters was that no one has really done this yet. Whilst I know that there are lots of case studies that can be trotted out etc, the reality on the ground is that outside of the true early adopters, it just has not gone “mass market” yet. SaaS adoption is the exception to this rule as this is more of an evolution of the previous ASP model, but PaaS and IaaS really have tiny market take up at present in the larger enterprises. My view is that SMEs are generally adopting quicker than larger enterprises.
• Risk – The audience did not feel particularly risk-averse. Again, I guess they had “self-selected” to attend, but I didn’t get any of that “corporate fear” paralysis sensation about data security etc that I’ve witnessed in some enterprises. So I felt like these guys were really going to make something happen over the next year or so, which I felt was particularly encouraging for both the industry and the UK economy.

One closing comment I would make about the event is that there were some points made during the day that I knew were just plain wrong, some of which fell into the FUD category. I managed to keep my mouth shut. Amazing as it may seem, I am also wrong on occasions, so I’m not having a pop here specifically – it’s just that even in the vendor/SI space we’re all still getting to grips with a disruptive technology and there is a lot of hyperbole and urban myths to be thrashed out and dispelled yet.