Application Development

I have just got round to catching up with some blog posts after holiday season and this one from my KCom colleague Rob Wells caught my eye. I particularly liked the word “phablet” to cover phones and tablets but I’m not sure it will catch on.

Rob makes a compelling case against a proliferation of Apps and for building ‘mobile optimised HTML’ which relies on an initial web server detection mechanism that checks the user agent (Rob says Operating System but I don’t think that’s what he really meant) and then directs the request to the right pages. This is absolutely the right approach given that more and more users are now finding your company websites on mobile than on full-scale laptop screens so immediately directing from “” to “” that is optimised for mobile should be a no-brainer.

I like the KCom portal approach of assuming controls are ‘touch-first’ as well. I overheard a conversation the other day where someone was wondering how long it would be until putting something on a web page like “CLICK” (PC or mouse-oriented) would die out in favour of “TOUCH” here. Perhaps somebody needs to invent a new word that covers both options. I tried out an online thesaurus for touch which offered “tap” or “hit” (and a few other slightly dodgy ones) but synonyms for click were even less helpful.

Anyway, I digress. Rob has made a very good argument for HTML(5) on browser being the target of choice but I would argue it is not always the best option.

Here’s just two reasons why:

1. You’re NOT always online. The Apps I use the most are the ones where I can use them offline (and occasionally synchronise data when the Interweb is available) for example EverNote, Strava or good old faithful email clients. I actually hate Apps that rely on Internet connection to function and avoid them wherever possible.

2. There isn’t very consistent or reliable support for HTML5 in many browsers yet and it’s just as hard to ensure your application works on web browsers as it is with native or “hybrid” Apps with lots of browser-specific catches that you need to work with. For the foreseeable future I can’t see pure web applications having enough functionality to compete with Apps which run natively on the popular handsets.

The nice thing about Hybrid platforms like IBM Worklight or Appcelerator is that you can still use a lot of the common HTML, CSS and JavaScript skills but write them to use APIs into the native features such as encrypted cache through JSONStore for offline storage and benefit from a little bit the best of both worlds. Worklight, which is now a leader in Gartner’s Magic Quadrant of Mobile Application development Platforms has the option to generate a mobile web version as well as version for iOS, Android, Windows 8 and BlackBerry so you can have your cake and eat it. It also addresses the problems highlighted in Rob’s blog of fragmentation of the different mobile platforms plus controlling versions and ensuring security.

There are advocates of both approaches and neither is really right or wrong – as always you need to take a decision based on your requirements, strategy and budget.

Today myself and four other Smarties attended Norfolk’s first Mobile Development Conference at the Hethel Engineering Centre, which is right next to where they make Lotus Cars.

Conference Room

There is an obvious tie-up between Hethel and Lotus given that the main presentations were held in the Colin Chapman room (founder of Lotus cars) where one of Ayrton Senna’s “99T” F1 cars was stuck to the wall!

Mobile Development is one of the most exciting and diverse areas in IT at the moment and this conference did very well to have a wide coverage from games developers like MonoGame to Tim Ferguson, Head of Digital at one of our customers AVIVA and their mobile app lessons learnt from their various innovations and experiments.

The keynote by Neil Garner of @Proxama resonated with me very much, both in his memories of tech from past years (Nokia 7110 first Wap phone) to his honest assessment of NFC and rebuttal of the doubters who don’t see NFC taking off now. The ARM Trustzone was highlighted by Neil as a key element in providing security for NFC applications. There are Contactless terminals everywhere now and 9 of the top 10 device manufacturers are signed up to support NFC – Apple is the odd one out but aren’t they always?

Our own @JamesElsey1986 later showed that NFC is more flexible and powerful than you think using Android. James later tweeted:

Source code/slides from my #NFC workshop  Feel free to ask questions / give feedback. Thanks for attending! #MobDevCon

Matt Lacey presented two sessions, his first on tips for developing for Windows 8 included some real gems which will help us with our tailoring of our cross-platform Apps to work well on the new Windows platforms. I agree with Matt, who worked on PhoneGap’s Windows integration code that you have to be knowledgeable and experienced in developing native Apps to be able to build successful cross-platform Apps. Luckily Smart421 have a whole Microsoft practice to help us Java-oriented types out with that. Read Matt’s blog for more info and his slides from his second presentation on monetising Apps.

I was first on to present after lunch and talked about our work delivering cross-platform mobile experiences with Worklight – my slides are now up on slideshare. There was a general theme at the conference that cross-platform tools are coming of age and the compromise of user experience and performance when compared to native development is far outweighed by the much faster and cheaper overall costs of App development and maintenance. I just about managed to demo the new Worklight 6 Studio IDE and Console. I am really liking the improved jQueryMobile integration and want to find time to check out the new App Centre tools and automated testing when I get the chance.

Ruth John (@rumyra) of O2′s “The Lab” gave a kitty and puppy-tastic presentation on FireFoxOS and why Telefonica have taken it up especially in the emerging South American markets – it’s free, works well on low-end handsets with the FireFox operating system built on top of the Gecko layer as is Android. It will be really interesting to see if this will catch on in the UK and European markets in these times of austerity where people are perhaps not quite ready to splash a few hundred every year on the latest iOS gadgets.

There was also a really enlightening “sponsor presentation” by Basho on the subject of reclaiming the terms web scale, big data, dev ops and how the NHS is using Riak’s open source technology.

Massive thanks to Naked Element (Paul and Marie) and everyone involved in setting up the event, thanks to Hethel for such a great venue, the sponsors for the delicious lunch and the attendees for their support and kind comments.

P.S. Welcome to twitter @CharlesBSimms :-)

After the polished video intro (screen shot below and catch the full key note here) Brad Anderson (Corporate VP at Microsoft) kicked off TechEd Europe here in Madrid by highlighting the opportunities that we have available to us as at the moment in IT. He talked about innovation, helping customers differentiate themselves and how Microsoft aim to make others great, so whilst I definitely agree that with him, it was the resonance with so much of what we do at Smart that I really liked.


Four areas of innovation were discussed around; People Centric IT, Building better Apps for people, Data, and the Cloud Platform. Aligning to these were new product release announcements including SQL 2014 CTP, VS2013 and TFS2013 CTP, Windows Server 2012 R2 and System Center 2012 R2. Better still, a partnership with Oracle was announced which means Oracle is fully supported on Azure and Hyper-V (and from what I gathered Oracle images ready to spin up in Azure).

One of the best parts (and I tweeted at the time – @wanty1975 btw) was a video of one of our customers (Aviva) referenced as an example of mobile and Azure development combining to provide an innovative solution for their customers. Why was this so good? Well partly because my old boss was in the video…but more so because seeing real world examples, and close to home too help inspire and make me realise the opportunities we have available to us.

TechEd Pres

So a brief list of new features mentioned in the key note that you should be aware of:

  • BYOD – two-factor authentication making it easy to add devices (a MS Surface in the demo) to AD domains using Windows 8.1 (not sure what the experience would be like for other devices though??). The simple process means files are sync’d, permissios granted to corporate data and Line of Business (LoB) applications downloaded onto the device accessible via a company portal-like homepage.
  • VS and TFS 2013 – ALM benefits throughout the develoment lifecycle but I really liked InRelease (following a recent acquisition by Microsoft) which provides workflow and approval mechanisms for moving builds through environments. I could see this really helping to take the pain away from tracking builds through environments and ensuring everyone knows what versions are where. From a dev perspective I also thought the new Code Lens and Code Map features looked really helpful. These come under the banner of Situational Awareness and are designed to reduce the time and effort needed by developers when context switching. Code Lens integrates with TFS 2013 to provide details within the code base, such as tests passing/failed, who it was last checked in by and any recent check-ins. It helps developers understand who has done what, where and why, much more simply than the previous version/compare routine that we’ve all experienced. Code Map provides a diagramatic view of code (similar to architecture modelling tools) but this can be viewed at runtime in debug mode, like a call stack, helping developers familiarise themselves with code much quicker and easier.
  • SQL 2014 – some huge performance gains achieved through in-memory OLTP and row based access rather than page, therefore reducing locking issues and increasing performance. SQL will suggest what tables are ‘hot’ and recommend moving them into memory (I’m assuming they’re persisted to disk to fairly frequently!!). I also picked up on the ability to replicate an existing on-prem DB to Azure, to provide HA and DR via a right-click menu option…didn’t see or hear any more evidence of this but sounds like a great feature. Also HDInsight and Hadoop linkage to Excel to provide visualisations and insight to Big Data.
  • Windows Server and System Center 2012 R2 – providing the tools to make Hybrid cloud simple and easy, with tooling to help sync data, create replicas for DR purposes and significant performance benefits for virtualised machines.

So there are obviously things that I’ve missed and lots more to talk about but hopefully this represents a fair summary. I’ll also be tweeting any further nuggets throughout the week.

Last week Smart421 once again was a sponsor at the Gartner Application Architecture, Development and Integration (AADI) Summit - this is the primary event in the UK Gartner calendar that covers integration/SOA, cloud computing, mobile, big data etc. Our marketing manager kept me rather busy with customer meetings throughout the event (!), but I still managed to catch some of the Gartner analyst sessions.

In the keynote, Andy Kyte made an interesting point about the barriers to SaaS adoption, in that organisations traditionally organise themselves along application lines (i.e. the System A team, the System B team…) rather than along domain lines (the “customer domain” team, the “product domain” team…). This relates nicely to our view of how organisations should develop their integration strategy over time. Anyway, although Andy didn’t use this term, this seemed to me to be yet another realisation of Conway’s Law, where individual team structures and the typical budget scraps over limited resources that they take part in put significant barriers in the way of the adoption of SaaS to replace existing legacy systems.

Ray Valdes presented on user experience design and gave a useful checklist of things to consider and the top 10 common mistakes to avoid. He made the distinction between design approaches led by intuition (“I know I’m right, trust me”) and evidence (“I’ve tried this and measured the outcome so I know it improves things”). I felt that the remedy to many of the top 10 mistakes lay in the application of the principles of the Lean Startup, e.g. test and learn, iterate quickly, validate your assumptions, collect metrics and feedback etc – although interestingly the intuition-based approach is more likely to lead to a innovation leap, whereas the evidence-based approach is more likely to lead to refinement and tuning of existing approaches but not such a large likelihood of a game-changing innovation.

On the Thursday there was the usual evening reception which was a lot of fun – this year we had some flair bartending on our stand from CocktailMaker – here’s an action shot… (Photo by Jim Templeton-Cross )

Gartner AADI 2013 stand

The cocktail guy did drop the bottle on the floor a few times following particularly ambitious throws but fortunately we didn’t burn the place down with any flammable liquids :). As you can see from the photo below, our team (Neil Miles, the MD, and two colleagues from Business Services) are very smiley after knocking back a few cocktails…and Red Hat’s marketing approach at the event clearly worked!

Gartner AADI 2013 smiles all round

Photo: Industrial backdrop by Pilarts  Dreamstime Stock Photos & Stock Free Images

Photo: Industrial backdrop by Pilarts Dreamstime Stock Photos & Stock Free Images

I’d like to propose a best practice for rolling out new features in a Service Oriented Architecture (SOA).

Traditionally, when we roll out a major new feature, we often end up causing a breaking change to the service. We’re then faced with a choice: (a) Force all our consumers to upgrade to the new version, and making all our consumers hate us, or (b) continue to support the old version of the service as well as the new, making only our own teams hate us. Suck it up, plan (b) is the better option, but try telling that to the guy having to patch fixes in three concurrent versions of a service.

Now, there are patterns that can help here (more on that another day), but they all still mean more work for everyone.

Also, when we first roll out a feature is exactly the moment we understand it least. We’ve got absolutely no idea how people will use it, nor whether it will even turn out to be useful. By baking the feature into a new major version of the service, we’re taking all our options away. The feature will be hard to remove if we decide it isn’t useful, and if we want to change how it works, we’re back into a major version upgrade again.

To my mind, good engineering is largely about keeping your options open. It’d be nice if we can try a new feature with a subset of consumers first, iterating quickly with just that subset, gradually adding more consumers as we get more confident.

Enter the Feature Flags pattern. Feature flags allow you to turn features on an off at a moment’s notice. At its most basic, a feature flag just turns a feature on or off for everyone at once, but the idea is often extended to allow turning on features for specific users, or collections of users. This allows you to roll out a new feature to consumers gradually, over an extended period.

So, here’s the proposal:

  • Allow consumers to pass a set of feature flags dictating which features they’d like enabled in the service.
  • Whenever you build a major new feature that would otherwise cause a breaking change, only enable it when the feature flag is passed.
  • If appropriate to your environment, control access to feature flags like you would to any other resource – e.g. you might want to restrict access in the early days to just a single consumer, making it easier to iterate.
  • Once we’re comfortable with a feature, it becomes publicly available – i.e. anyone can toggle the flag.
  • Every so often (e.g. once every couple of years), create a new major version of the service, refactoring it to include popular, battle tested features by default. Also, take this as an opportunity to clean out the cupboard and abandon any features that aren’t well used.

What do you think? Comments and thoughts very welcome…


Please remember to Rate and Like this post.  If you can, please leave a Comment.

Wow! SyncIpswich’s second meetup and around 80 people crammed into the Eastern Enterprise Hub in the James Hehir Building at University Campus Suffolk. Many of the attendees were working for local behemoths like BT but there were also a good mix of bootstrappers, Start Ups and tech entrepreneurs with all kinds of backgrounds (even spotted a Chartered Accountant).

Organisers Carl Farmer (@CarlFarmer), supported by Anders Fisher (@atleastimtrying) and others have done a great job with SyncIpswich, which we are proud to sponsor. The focus of this meetup was on building software quickly with good practices as well as a nice introduction to the Windows Azure Cloud.

Talk no 1. Continuous Delivery

The first presentation by Chris O’Dell from 7digital (@ChrisAnnOdell) described how Agile practices (CI, Kanban, etc) combined with their architectural evolution to SOA have reduced code to deploy times to half a day  at 7digital.  And, by the sound of it, makes their developers more productive by getting away from “DLL Hell” that used to be the bane of any Microsoft Windows developer’s life towards a loosely-coupled set of services and a public API.

7Digital Logo

Chris raised some really interesting points around developing small fine-grained service components – not being that familiar with .Net myself this seemed to be similar to what we are doing in the Java world with OSGi and Service Component Architecture. I do like the policy of developing new features on the trunk (no feature branches) but making good use of feature flags rather than old-fashioned branch & merge.

They are also using Git for the code version control and Chris showed the inversion of the classic Unit Test, Acceptance Test, QA triangle. Some in our own organisation are raising question marks about the usefulness of very granular unit tests so the approach taken by 7digital of increasing the number of Unit tests is interesting.

There were a lot of questions from the floor, I was particularly interested in how the small kanban teams (about 6 or 7 members in 5 or 6 teams I think) interact when there are common services. This is a key problem that us SOA architects need to get right to get the best value on services. Feature Flags is something that we’ve also thought about in the context of simplifying application testing by, for example, switching off authentication for functional testing.

It’s great to see a company like 7digital competing successfully with iTunes and Amazon in the digital music space. I’ll be checking out their API (and their JLS back catalogue !) in more detail this weekend.

Richard Astbury AzureTalk no 2. Starting out in Azure

The second talk of the night was by Richard Astbury (@richorama) of Two10 Degrees ( @two10degrees). Richard gave a nice introduction to Cloud computing and in particular using Microsoft Windows Azure, showing a picture of a MS data centre under construction, which was something I haven’t ever seen before. I think it really brought home the sheer scale and commodity nature of the Cloud and these facilities being full of containers of kit that is just thrown away or recycled when it stops working.

Building a website on Windows Azure from scratch can use a few main pre-canned routes like the obvious “Website”, “Virtual Machine” and “Cloud Service”.

And it now includes a “Mobile Service” which is of particular interest to me. Sadly, I didn’t have time to chat to Richard about this but it’s on my “To Do” list to get a Hello Smartie mobile service up and running. In fairness, Richard did do two masterful demos for Website, including a node.js based site which he even launched from his home computer (a Raspberry Pi no less). As Carl tweeted:

Deploying to Azure from a remote RaspberryPi at home… Impressive stuff from @richorama !

— SyncIpswich (@SyncIpswich) April 25, 2013

Well done to the people of Ipswich for turning out and drinking all the sponsored free beer!
SyncIpswich will run and run.

Please Rate and Like this blog.  If you can, please leave a Comment.

Last night, I was one of 6 presenters who gave 5 minute lightning presentations at the monthly Front End Suffolk meeting in Ipswich.

I love the lightning presentation format as you get a really broad range of presentation topics and styles in a short burst – I first came across it at CloudCamp some years ago and we’ve used it internally in Smart421. Presentations covered Jasmine for JavaScript unit testing, use of make and accessibility considerations with links – a really good mix of thought provoking topics.  Thanks to Anders Fisher ( @atleastimtrying ) for organising the event and for the invitation to present. Great group Anders..

At the previous month’s FESuffolk meeting Paul Hutson presented on KineticJS (see his presentation here) and that got me thinking/comparing it with the work I’d done with YUI, especially related to collision detection – so I foolishly stepped up and offered to present on the topic. Here’s my presentation on SlideShare…

Please Rate and Like this blog post.  If you can, please leave a Comment.


Which Essex boy doesn’t want Titanium Alloys?
Photo © Maxpro Dreamstime Stock Photos & Stock Free Images

Introduction to Appcelerator

I’ve had my eye on Appcelerator and their Titanium mobile development platform product for some time. So I grabbed the opportunity to venture north of the [Suffolk] border to the “SyncFocus” a SyncNorwich spin-off event that was held in the Garden House Pub, upstairs room, in Norwich last night (20th March).  So, reflecting on my rough notes, I’m posting this blog with a few random thoughts thrown in and some links to further information for those interested in javascript-based mobile developments.

This SyncFocus featured as guest speaker non other than Boydlee.  For those who haven’t heard of Boydlee, he is a mobile application development guru and specialises in the Titanium platform who, like Beyoncé, is now so famous he doesn’t need to use his surname :-) .

My “executive summary” is that Titanium is a serious productivity tool for people who can write “well written” javascript code. It is in competition with the HTML5-based approach of PhoneGap, and does not (yet) have as many surrounding services as IBM Worklight which is squarely aimed at the big Enterprise market and is not as broad in its coverage of operating systems and devices (yet). It does however, have a thriving developer community and a very attractively priced (i.e. “FREE” cloud service). It does appear to be very easy for a moderately skilled javascript developer to create apps but it’s probably just as easy to create a horrendous mess so the newest approach is to supply a framework as the well-known MVC pattern through “Alloys” – which Essex boy doesn’t want Titanium Alloys on their racer?

Main points:

  • Titanium is purely JavaScript library coding against the Titanium api, the platform compiles the source to native Objective-C or Java code for the iOS and Android Apps and can easily be linked with the Xcode or Android SDK to complete the builds.
  • Links with emulators for iPhone, iPad and Android
  • Appcelerator has over 400,000 registered developers worldwide (a counter on their home page tells me it is just under 430,000 today)
  • The libraries themselves are “Free and open source”, available on GitHub
  • However, the Titanium studio is not free, and you can take the Appcelerator Paid support
  • An estimated one third of cross- platform apps are built on titanium
  • Claims to also support BlackBerry 10, but Windows 8 is not available until later this year

One really nice feature is a very large “Module marketplace” which is mostly compiled binary code aimed at one native operating system or the other that can be downloaded and incorporated into the build – for example something to integrate to native libraries, plugins, e.g. Barcode scanning

A lot of those modules also have their source code on GitHub, building the community even more and improving the eco-system. This appears to be thriving and also includes some javascript extensions as well as the compiled native iOS and Android libs.

Appcelerator Mobile Service (ACS)

ACS is an interesting move to give a free level of Mobile (application) Back-end as a Service (MBaaS) to developers of services, running on AWS.

This includes 20GB storage, 5m push, 5m api and 20 pre built common services, including key-value pairs (similar to AWS Simple DB), Ratings, Geo services, media, including thumbnails, Social integration (Facebook) and storage of custom objects, which I think was based on Mongo DB as a service. There is certainly capability to link directly with Node and Mongo backends and simple stuff like SMTP integration (again similar to AWS’s Simple Email Service “SES”).

Coding practices

Boydlee recommended looking into Common.js best practice for relative newbies to Javascript. Other frameworks are available, like backbone, require.js and one of his blog posts has recommended the javascript patterns book by Stoyan Stefanov.

There are also free books online available on styling, including best practices to cascade for tablets so you don’t just get an enlarged phone experience that sucks.

The CSS approach also over-rides for different devices, so it’s not always best to stick with pure javascript for everything, however, for those who really are blinkered by javascript there is a CSS to JSON converter library.

Boydlee’s books are on PACKT or there are kindle versions. Boydlee’s best practices book is one I will be getting for sure.

Finally, the newest developments in Appcelerator will probably be using Alloy which includes full Model View Controller (MVC) frameworks (again, all built on well-known javascript frameworks) that claims to speed up App development by another order of magnitude.

Comparing it with the IBM Mobile First offering the coverage of mobile O/S is much wider in Worklight (Windows 8 and BB 10 were available very soon after launch), the adaptor environment, the upgrade management and enterprise app centre seem more developed and you have the whole extra wrappers of security, analytics and so on within IBM’s Mobile offering. The customer list on Appcelerator’s website is mighty impressive for consumer-oriented Apps but the ability to integrate with Enterprise back-ends is less of a focus, which is why in our case we’re focusing on IBM for Enterprise mobility but Appcelerator is certainly worth considering.

Please Like and Rate this post.  If you can, please leave a Comment.


In my previous blog post on using AWS CloudFormation to provision a Centos based environment I mentioned how at first the JSON syntax used within CloudFormation templates can be a little daunting, especially for people with limited scripting experience.

At the time of writing the blog I was using a combination of   Notepad++ with the JSON Viewer Plug-In to create and maintain my CloudFormation templates. One of the problems I had with this approach was that the templates are treated as pretty basic text files by Notepad++ and the JSON Viewer Plug-In only checks the base JSON syntax, so there is no validation of the objects defined in the JSON to ensure that the correct properties are being creating to allow the template to form a valid CloudFormation stack.

As an early Christmas present to anyone working with CloudFormation AWS recently announced the introduction of a new CloudFormation editor as part of the AWS Toolkits for Visual Studio and Eclipse. I have now had a chance to download and experiment with the Visual Studio version of this and am really impressed with how much easier it makes the creation and maintenance of CloudFormation templates.

As a .Net developer I have grown used to regularly relying upon IntelliSense features to aid with code creation, particularly when it comes to accessing and updating object properties. The CloudFormation editor provides this for the objects defined within a template as well as code snippets for all the object types you might want to define within your template. This greatly reduces the number of errors caused by ‘finger problems’ that used to occur when creating a template by hand.

The other really useful feature of the editor is the ability to estimate the costs of a stack before creating it. In the past when quoting for customers we have tended to pull together a design, plug the figures into the AWS Simple Monthly Calculator and then once we have received the go ahead provision the environment. With the ability to create costs from a template we are now looking at generating a base template at the design phase and then costing and building from this, which should help with the speed of environment creation and clarity around the prices for altering the design.

Based on our experiences so far  it’s a big thank-you to AWS Santa and his helper elves for the early Christmas present of the CloudFormation Editor and we are all looking forward to further presents from AWS  throughout next year :-)

DATABASE at Postmasters, March 2009DATABASE at Postmasters, March 2009 to Michael Mandiberg

“NoSQL” is an unfortunate term for the currently hype around non-relational database systems. Many of the ideas presented at the NoSQL Roadshow in London of the new wave of different databases are not new. More than one presenter used the term preSQL, and during a break Brian Bulkowski from Aerospike explained that Oracle had created these ideas many times over, but had not considered them commercially worthwhile. What has changed in recent years though is the business need. There are new problems to solve such as online and mobile gaming and advert-serving that require users to trade-off consistency, availability, and partition tolerance.

For example, Amazon require a horizontally scalable system (partition tolerance), so that when, and it can be any time of day, a customer adds an item to their shopping cart, it is captured in the database (availability). They are less concerned about the time it takes for this to filter through their other data stores (eventual consistency), or how long it takes to fulfil the order. In the mid-2000s Amazon realised that the current relational database were not meeting their needs and they created Dynamo, a highly-available and scalable technology for key-value storage.

Papers by Amazon on Dynamo, and Google on their technology BigTable, were a major contributor to the current early adopter market. Today there are many competitor NoSQL products, including the aggregate-oriented databases (key-value stores, document stores and column stores), and graph databases.

Key-value stores are the simplest store type, with keys mapping to binary objects. They allow low-latency writes and scale easily across multiple servers, but can only offer single key/value access. David Dawson and Marcus Kern of MIG gave an example of using the key-value store Riak as a persistence store for a bespoke queuing system in their SMS gateway product. The biggest difficulty was finding a way to simplify the retrieval of messages in the event of a node failure; their solution involved the use of predictable keys.

Column stores manage structured data, stored in columns, with multiple-attribute access. Apache Cassandra, originally developed by Facebook, is a well known example. These are also optimised for writes and used for high throughput uses such as activity feeds or message queues.

In document stores keys are unique references to “documents” which encapsulate and encode data in a standard format. These database systems hold hierarchal data structures that reduce the need for table joins and allow for variety and evolving schemas. Akmal B. Chaudhri of IBM presented his investigations into the popularity of the various NoSQL offerings, showing that MongoDB, a document store for JSON documents, is leading the way. Some of our Smart consultants are using MongoDB from an application development perspective and are very positive about their experience so far. It will be interesting to follow these projects and understand the effects on administration, support and future change. What impact will the schemaless nature of the database have?

Graph databases use the concept of nodes and edges to store information about entities and the relationships between them. Jim Webber of Neo Technolgy gave an example of the use of the open source Neo4j graph database in modelling the relationships in Doctor Who.

Each type of database system has its own strengths and weaknesses, and the reality is that NoSQL databases will only be used as part of a solution. Choosing different data storage techonologies for different persistence needs has been termed polyglot persistence. Every example provided at the roadshow included relational database systems alongside NoSQL technology. Wes Biggs of Adfonics, an independent advertising marketplace, explained the architecture of their solution for buying advertising on mobile devices on behalf of advertising agencies. They use the relational MySQL with data on hard drives for long running information such as campaign details, MySQL Cluster with data on flash drives for aggregate instructions such as user details, and Aerospike with data in RAM for raw instructions such as the inflight data.

The overall message from the NoSQL Roadshow was that this is still very early days for NoSQL database systems, and it is not yet clear whether the future popularity will be closer to that of OO or relational databases.

Many presenters offered words of caution. Security is a big issue and many of the major NoSQL database systems must be run in a trusted environment. Wes Biggs discussed the issue of the huge number of vendors and the lack of evidence for their claims. At this moment any system choices are basically faith based, and should only be taken if someone else has used it for a similar use case; they were burned with a few early choices for a key/value store before they settled on Aerospike.

The Fusion-io presentation on their new directFS filesystem for flash drivers was only one example with obsession with performance, and the importance of the hardware architecture.

At the moment there are some exciting potentials for NoSQL technology, but anyone getting involved at this stage will be making some brave choices. It will be fascinating to see how the market shakes out over the next few months and years and I’ll be following with interest.

Update: 18 Dec 2012 – interested to learn more about MongoDB ? Click here.

Please Like and Rate this blog. If you can, please leave a Comment.

Next Page »


Get every new post delivered to your Inbox.

Join 1,122 other followers