Video: Setting up a continuous integration environment


I’ve blogged a lot about how CodeGear uses $g(Continuous Integration) (CI) in it’s development process for Delphi/BDS. What want to talk about here is how you can use these same tools for your development process.

First, is the issue of $g(version control) and for that we use $g(Subversion) otherwise known as SVN. SVN is open source and fortunately there is a convenient “1 click setup” described as follows:

Svn1ClickSetup takes a user through the steps necessary to install the Subversion command-line utilities and TortoiseSVN, as well as creating a repository and initial project.

This setup really lives up to it’s name so if you’re looking for a fast and easy way to get started with some great source control software this is what you’re looking for. The install includes the $g(TortioseSVN) client which is an SVN client that’s implemented as a $g(Windows Shell Extension) and is what the majority of developers at CodeGear use.

A Continuous Integration Server

That may sound a bit more daunting than perhaps it should but basically it’s the software that pulls from your repository, builds your project and reports any errors. The Delphi development team uses CruiseControl.NET server and CCTray for client side notifications of build failures both of which are open source projects and even though it has “.NET” on the end of its name the server isn’t restricted to building .NET only projects.

Once again the installation is very straight forward and if you have IIS installed you’ll even be able to monitor your CI server via a browser including forcing a new build.

Ok, now that you’ve read all about it have a look at the video. Or can download it here.

Here is the ccnet.config file used in the video, and no, “steve” is not my normal password. 🙂

[UPDATED: April 23, 2007] Fixed video and .zip file download links.

23 thoughts on “Video: Setting up a continuous integration environment

  1. thank you steve,
    very nice and very useful video
    me want more topics on agile , automation, testing

  2. Hi Steve,
    Can you add a zip download option for your video, so I can watch it offline.
    Also your link to the ccnet.config file does not work.
    Cheers Stu

  3. Wow, was that ever needlessly complex, almost as if the concept of integrated software meant running a bunch of differnt software at the same time and as if Delphi didn’t have a build engine installed.
    Forgive me if I completely miss the point, but what the heck is the point here? I assume that you would have a central SVN and a central CC server monitoring it trying to decide if the 12 differnt things checked in by 12 different people can all compile together and hope people can sort their error messages out from the missmash, or just get lucky and be the only check in so only their error messages popup?
    I assume a larger project would build less frequently, and this means that more people could check in between scans increasing the possibility of a mishmash of errors, assuming that your code even got the chance to compile because of someone else’s errors.
    Ok, let’s set that aside. I think that this nicely illustrates one of the LONGEST standing problems about Delphi and so called version control software. Version control works at the project level only. If Project X depends on components a & b, you had better hope your build server and all developers all upgrade at the same time, or seriously bizarre things could start to happen. (including apps that compile, but won’t run since the DFM is validated only at runtime)
    Worse, if you have to roll back to last weeks work, if you use external dependencies like third party packages, you STILL have to track updates and manage them independently to ensure that the multiple projects are all synced properly. Oh, and hope that the effect it has on your IDE isn’t negative in the extreme. (if you haven’t seen what a stray, version mismatched DCU,DCP or BPL can do, just wait, it’s a party!)
    Bolting together a bunch of software original written for UNIX C apps, ported to windows and then adding a layer of automated gooeyness just doesn’t make sense to me for Delphi projects. I know from experience that 2 week old code can be completely uncompilable against today’s components. Delphi needs a BETTER, FULLY integerated solution that takes the full scope of delphi project into account.
    Heck, I have a production system that involves in excess of 100 component packages and over a dozen apps (that I am responsible for, the other developer brings more apps to the mix). This approach would fail to scale to that almost immediately. I would spend serious amounts of time just trying to remember what to check in with explorer, and probably fail and loose things on the next check out.
    So far, the only solution I have found is to push the entire dev environment into a VM. Source, compiler and components are all bound together in to a fully versioned image that can be saved and pulled out again at a moment’s notice and actually expect it to compile almost immediately (gotta boot the VM and start the IDE). Merges are done through isolating primary areas of responsibility and WinMerge to check for missed details.
    And I can’t really call what has grown to 12gig drive images a "version control system" with a straight face. Thankfully, drive space is dirt cheap, because even DVD backups have gotten too unweildly.
    Basically, I feel that Delphi needs its own unique version control system that takes all this into account, and this video just unscores my feeling that what is available falls far short of that goal.

  4. I do believe that hit rant proportions. Sorry, version control is obviously still one of my hot buttons.

  5. Steve,
    First of all thanks for the article, I’ve been looking for something like this for a while that is specific to Delphi. I have recently built a CCNet server and I’m trying to integrate "WAnt" (, DUnit and Starteam. What I would love to see is if someone could modify CCNet to provide the "extras" for DUnit and "WAnt" that CCNet already has for NUnit and Nant (the .Net build tools). Not that you cant run Dunit and WAnt in CCNet, it just would be nice for them to have the same "built in" functionality. WAnt is a very good tool when you figure out how to use it (it has no real documentation to speak of) and I think a build system that integrated DUnit and WAnt with a source control system of your choice would be a great tool for Codegear to pimp "hint..hint".. lol.
    Anyway, again, thanks for the article. We want more Agile, more Continuous integration! keep it coming.

  6. C Johnson,
    What your comment points out to me is that more education on how to use these tools to build an effective CI environment is probably needed. While many times I tend to agree with your comments I simply couldn’t disagree with you more in this case particluarly with the "Delphi needs its own unique version control". Seriously, that’s crazy talk. Can I recommend you try these tools before you head to rantsville. These aren’t be-all-end-all solutions, they’re tools and used correctly can solve huge problems very effectively. SVN is *far* better than you think. It’s pretty much unanimous on the Delphi R&D team that SVN just rocks.
    Last thing, regarding your second paragraph which starts "Forgive me if…". Well, let’s further complicate the situation with compiling the BDS studio for four different spoken languages, using a prebuilt C++ compiler delivered in binary form from it’s own SVN repository and a Delphi compiler which is built during the build process with third party software integrated into the mix as well as far more than 12 developers working around the world 24 hours a day with automated DUnit and GUI tests that launch and drive the IDE and trigger integration builds at the successful completion of all this. Yeah, SVN and CruiseControl.NET handle this no problem. And, btw, sorting out error messages is pretty obvious when their detected within an hour after checkin.
    Also with SVN, there is no "trying to remember what to check in" you just select "Check for modifications" and it will tell you what’s modified locally as well as remotely it’s cake.
    Ok, rather than try and address all these issues here with a comment I’ll make an effort to see if I can’t address at least some of them over time with actual blog posts. In fact, some of the post I’ve written previously on the topic of Agile would apply here.
    Oh yeah, get yourself a copy of Araxis Merge.

  7. Thank you. As always very useful and interesting information. Keep going!
    One question: why don’t you use StarTeam? CodeGear move StarTeam but use SVN 🙂 Maybe I do not understand something 🙂 I am not very advanced in source versioning so far, that’s why this topic is very interesting for me.
    Thank you!

  8. Steve -> I’ll do my best to keep an open mind as you lay out more entries on this topic.
    I am concerned about 2 points, however. First is the check for modifications, but I’ll do some playing and see how it actually handles my scenarios (multiple component projects in multiple directories etc) and the second is the "if found within an hour" basically means you hope that someone else’s changes don’t blow up the compile process for too long before you get a chance to get see your stuff succeed/fail.
    I will add that I’ve seen a few delphi features that seem to work great for the codegear team, but seem to have some serious issues anywhere else. One that jumps immediately to mind is the LIB Version on packages. If you only ever have to support a single version of delphi at a time then it works fine, but otherwise there are definite concurancy issues. ( Caused by the fact that every version till now shared the same basename.cfg leading to serious problems supporting multiple language versions side by side in the same directory which is pretty common for component writers, in fact, the work around involves a different directory for every version of the compiler to store the package files in, which really seems to just make everything harder to maintain )
    That said, I’ll keep my eyes peeled for future posts.

  9. > (including apps that compile, but won’t run since the DFM is validated only
    > at runtime)
    For this we (at work) use a little tool that compares the DFMs with the form/frame/datamodule classes. This tool is integrated into our build system. So every time we build our projects the DFM check will fail if there is a mismatch with the class and its base classes.

  10. Steve,
    Thanks for the post. Rather timely for us at work.
    Any more videos planned like this?
    Also, I’ve tried sending you a email using the "Contact Me" link, but I keep on getting sent to the error page.

  11. Hi Nicholas,
    Thanks for the feedback. I don’t know about more videos at this point because they do take quite a bit of time but I would like to continue posting about CI in general. There is certainly a lot that could be covered here as is evident with C Johnson’s comments above.
    Regarding the Contact me link I’ve been getting email from people and I just sent myself email from FF, IE6 and IE7 and they all worked. If you email me your IP address I can try to look it up in my logs and see if I can find an error. My email is strefethen at codegear dot com.

  12. Steve,
    Why Subversion and not the in-house solution (a.k.a StarTeam)? Just curious

  13. Fabricio,
    We used to run StarTeam but the lack of "atomic checkins" became a real show stopper. I believe StarTeam supports atomic checkins now but don’t quote me on that. Additionally, the maintenance requirements on the server were quite high and when CodeGear split from Borland it became another issue that we were interested in mitigating. SVN was more or less an obvious choice at a very attractive price. We’ve had practically zero maintenance issues other than running out of disk space on the server (once). A number of us had been experimenting with CVS and SVN at the time for personal projects so it helped that the team had it’s own organic support solution though it’s turned out that that wasn’t necessary. SVN has been rock solid for us over the past year.

  14. So now that everyone at codegear is using Subversion(which rocks by the way) how long before we see it in the Delphi IDE?
    It works so much better than StarTeam.

  15. Steve,
    I didn’t know supported delphi compilation. But it sure gives some possibilities. I guess I will run it in some kind of test environment before I inform my boss about it, so we can use it at work. Thanks for the video.

  16. Steve,
    how do you manage IDE packages in SVN repository? Do you have separate repositories for each package or you have only one repository for all packages and all editions of Delphi?

  17. Have you evaluated Perforce as an alternative to SVN? We’ve been using it for almost 6 years now, and it’s still going strong.
    One thing I’d like to know is, how SVN handles branches, and do you actually use branches at CodeGear?
    At work, we use branches of our code; One main branch (in which we never code directly), all releases are seperate branches (in which we only do hotfixes) and a lot of sub-branches (in which the actual work is being done). Only when issues are completely finished and tested, they are integrated into the main branch – this gives us the huge advantage that we can create a release at any time, and we can easily postpone issues when the need arises.
    The cost is, that we have to do manual integrations between the main branch and the other branches (which can sometimes be a bit of a burden), and we need to bring the sub-branches up-to-date once in a while. Luckily, Perforce helps a lot in this regard.
    Every night we run a build-script for all branches that have changes in them since their last build. (We also create the manuals and setup files, among other things.)
    How does this workflow relate to Continuous Integration? Or is it the same and did I miss something here?

  18. Hi Patrick,
    We evaluated Perforce several years ago and I can’t remember the reasons why we didn’t choose it though this rings a bell (from the Perforce website):
    Users 1-20 @ $800 per user = $16,000
    Users 21-50 @ $750 per user = $22,500
    Users 51-85 @ $700 per user = $24,500
    Total = $63,000 for one year
    Then another 160 per user per year.
    Yikes! We would probably need something like 50 seats. Moving to SVN has been a great experience for our team in fact, we just had our Delphi 2007 post-mortem meeting and both CruiseControl.NET and Subversion where listed on the "what went well" side which is the first time I can recall our source control tool being on that list.
    For details on branching in SVN see:
    We use a very similar technique WRT branching here and SVN has made that a breeze.
    Regarding your Continuous Integration question a nightly build is ok but what if a change is checked in that breaks the build right after that run? You wouldn’t pick that up until the following night. CruiseControl.NET triggers a build anytime the repository is changed so we typically know within an hour if there is a problem. Additionally, CruiseControl.NET will send notifications via email and a tray application when the build fails. It also keeps logs for each build which are available via a web page so you can quickly view a history of your tree(s).

Comments are closed.