Anyone doing modern enterprise software engineering today will be practicing or partially practicing Continuous Integration, otherwise known as CI.
If you have never heard of CI before, Martin Fowler has a really good post on the subject here: http://martinfowler.com/articles/continuousIntegration.html
Just quickly, CI really comes down to a few main key points – at a minimum (some of which lifted from Fowlers post above):
- Automate the build
- Make your build self testing
- Everyone commits/check’s in to trunk every day
- Every commit/check in should build trunk on an integration machine
- Support for buddy builds
- Keep the build fast
- Must be able to run a local command-line single build that does everything that represents the build machines process
As far as I am concerned those 7 items are mandatory. There are of course many more that make up a really good CI strategy which I’ll go into in a future post but for now, lets concentrate on those 7 items.
I’m not going to go into detail into each item, but will pick out the ones I think are the most important.
2. Make your build self testing
The thing is everyone seems to know about CI now, which is good. But many are still getting it wrong.
Number 2 on the list need explaining in order to get it right. I’ve seen so many large companies get this wrong, let me explain.
There are normally two types of automated testing:
- Unit testing (by no means should the component being tested go down a rabbit hole and test a whole bunch of other components, these dependencies should be isolated out)
- Integration testing or acceptance testing
So if unit testing is done right i.e. they are *not* integration testing and the tests test the smallest testable unit, then unit testing honors point 6 (keep the build fast) on the above list. Who cares whether the build is fast? well you normally only have a finite number of build servers and on those servers you have a finite number of CPU’s and RAM which means a limited number of agents. When all your agents are busy serving multiple simultaneous check-in’s, builds get queued, the more queued builds the longer it takes to see feedback from a check-in/commit.
If you don’t have a buddy build system (point 5 on the list – I’ll explain what a buddy build is in a minute), imagine what can happen when you get lots of queued builds, if 1 fails, then it is very likely all the following builds will fail. This means someone has to sort out the mess. This mess then break downs the benefits of CI.
So going back to my original point, keep the build fast is very important that often gets missed or not really thought about too much. I often see unit tests doing too much meaning that they take too long to execute. I often see unit tests being a mix between integration tests (like BDD/acceptance tests) and unit tests more like a hybrid. Unit tests should only test a single class – where possible, this keeps the build very fast.
But then what about acceptance/integration/BDD tests? normally such tests will execute end-to-end i.e. from a UI all the way to the database or downstream system and back. If you practice BDD you’ll have these tests for all your use cases main and alternate flows. So often you will have hundreds of these tests for a typical enterprise large system. So with that, these tests will take much longer to run due to the fact the tests are testing the full stack. Sometimes the downstream system will be on another network perhaps communication to it is via a IPsec VPN tunnel to the other side of the world. Or could be over the internet. Whatever it might be, incorporating these tests with the unit tests is not a good idea.
So when should you run these long running integration tests and when should you run unit tests? Integration tests should normally run nightly i.e. like 02:00am when the build servers are quiet and don’t have much activity. Unit tests should run on every check-in/commit during the day for constant feedback.
My CI tool of choice is Microsoft Team Build (comes with Team Foundation Server). In Team Build 2010 it allows you to easily setup controllers so I often use one controller for integration and one for CI. These controllers can delegate to n number of build servers. So with this kind of configuration, I normally have quite a meaty set of CI build servers (for running unit tests on each check-in) then have a less powered build server for deploying code and running integration tests.
The power of this is such that the whole daily build activities is fast, thus satisfying the fast build and constant feedback requirement. Developers can also start an integration build anytime they wish during the day which will not affect the day-to-day running of a software development team.
5. Support for Buddy Builds
What the heck is a buddy build I hear you say! I’m not sure if this is just a Microsoft term or not that has come from Redmond. But a buddy build is a way of ensuring your changes will integrate with the version on the server before your changes are actually committed into the branch. So how do you do that? in the old days before software products did this, you would send your changes to a buddy who would merge your changes with his, then he would attempt to compile/run tests etc before you committed your changes.
In TFS 2010, Microsoft has introduced the concept of a gated check-in. So when you check-in, you actually check-in as a shelveset (not to the main branch) then Team Build will execute the build scripts against your shelveset first, if it succeeds, then your changes will be committed to source control – hence it is impossible to break the build!
I’m not sure about other build server technologies like, Jenkins, TeamCity, Hudson etc but if you want to read more about TFSÂ 2010 gated check-in, see here: http://blogs.msdn.com/b/patcarna/archive/2009/06/29/an-introduction-to-gated-check-in.aspx
7. Must be able to run a local command-line single build that does everything that represents the build machines process
Now this is very important. I see all to often that there are no command-line builds at all! and if there are some, you sometimes have to run more than one to get a complete picture of whether your check in will succeed on the server or not.
The benefits of having a local command build are huge, here are a few reasons:
- It allows you to get a local copy of the solution running, i.e. setting up a web server, adding users setting security permissions, deploying web sites into a web server
- Have one single command-line build script batch file or power shell script that does everything your check-in will do, i.e. compile, run code analysis, run code style rules, run tests etc
- Enables faster development,Â geeks love command-line tooling!
If you have read this far, congratulations! until next time..