I've been working on a custom CCNET implementation to handle EDI invoicing which is nearly complete and one of the things I needed was FTP download support. In the project I'm working there is a utility that makes use of edtFTPnet, from Enterprise Distributed Technologies, an Open Source C# FTP library licensed under LGPL. It seems to work reasonably well though the authors are admittedly "love the 'internal'" keyword thus making extensibility difficult, if not impossible, short of forking the codebase. Anyway, I've been using the library and it seems to work just fine. Btw, I've submitted my suggested changes so hopefully they'll recognize the benefits and incorporate them.

Writing Plugins for CruiseControl.NET

Anyway, what I needed as a mechanism that would poll an FTP server looking for new files to be downloaded which sounded like a reasonable thing for CruiseControl.NET. Of course, once files are downloaded additional processing is done so it fits with CI quite nicely. Fortunately, CruiseControl.NET has a nice plugin architecture that makes this sort of thing easy.

Creating a Plugin Assembly

The first step is to create a new Class Library following the naming convention ccnet.*.plugin.dll and add the following assembly references:

• edtFtpnet
• Thoughtworks.CruiseControl.Core.dll
• NetReflector.dll

And add the following to your using statements:

• System.Collections
• EnterpriseDT.Net.Ftp
• Exortech.NetReflector
• ThoughtWorks.CruiseControl.Core
• ThoughtWorks.CruiseControl.Core.Util

Your assembly should appear as follows (click the image for a larger version):

Adding ISourceControl

The next step is to add the ISourceControl interface and provide the necessary attribute which will declare the name used to identify your Source Control Provider.

[ReflectorType("ftpscc")]
public class FTP: ISourceControl

The ReflectorType attribute comes from the NetReflector assembly and is located in the Exortech.NetReflector namespace. The value provided to the attribute is the name which will be used in ccnet.config as follows:

<sourcecontrol name="ftpscc"></sourcecontrol>

Adding FTP Specific Properties

Next, add properties for the FTP connection and once again we'll use attributes from Exortech.NetReflector making it easy to declare settings from ccnet.config:

[ReflectorProperty("deleteRemoteFiles", Required = false)]
public bool DeleteRemoteFiles = true;

[ReflectorProperty("fileMask")]
public string FileMask;

[ReflectorProperty("localPath")]
public string LocalPath;

[ReflectorProperty("password")]
public string Password;

[ReflectorProperty("remoteHost")]
public string RemoteHost;

[ReflectorProperty("timeOut", Required = false)]
public int TimeOut = 60;

[ReflectorProperty("userName")]
public string UserName;

[ReflectorProperty("remoteDir", Required = false)]
public string RemoteDir = "/";

These properties indicate settings for the FTP connection as well as the location where the files should be placed after being downloaded. Additionally, there is a flag which indicates if the downloaded file should be deleted from the server.

Again, simply adding the ReflectorProperty attribute will cause these members to be populated with the data from your <sourcecontrol name="ftpscc"> tag in ccnet.config.

Implementing ISourceControl

Next, implement the ISourceControl interface on your class. Of course, the easiest way (in VS.NET) to stub out the necessary methods is via the context menu with the caret sitting on ISourceControl:

This will give you five new methods though I'll focus on the following two since that's all you really need to implement here:

public Modification[] GetModifications(IIntegrationResult from, IIntegrationResult to)

public void GetSource(IIntegrationResult result)

In GetModifications we need to return an array which contains information about the files to be downloaded and in GetSource actually fetch the files. One caveat here is we use DateTime.Now as the modified time so that CCNET will consider the file as a "new" change. So, GetModifications will look like this:

 1 public Modification[] GetModifications(IIntegrationResult from, IIntegrationResult to)
 2 {
 3     ArrayList mods = new ArrayList();
 4     Connect();
 5     try
 6     {
 7         Log.Debug("Fetching list of files that match: " + FileMask);
 8         FTPFile[] mModifiedFiles = ftp.DirDetails(FileMask);
 9         Modification mod;
10         int i = 0;
11         foreach (FTPFile f in mModifiedFiles)
12         {
13             mod = new Modification();
14             mod.Type = "Added";
15             mod.FileName = f.Name;
16             mod.ChangeNumber = i;
17             mod.ModifiedTime = DateTime.Today;
18             mods.Add(mod);
19             i++;
20         }
21         return (Modification[])mods.ToArray(typeof(Modification));
22     }
23     finally
24     {
25         // Close the connection if there are no files to process
26         if (ftp.IsConnected && mods.Count == 0)
27             ftp.Quit();
28     }
29 }
30 

GetSource handles downloading files from the server and then deleting them:

 1 public void GetSource(IIntegrationResult result)
 2 {
 3     try
 4     {
 5         foreach (Modification m in result.Modifications)
 6         {
 7             Log.Debug("Fetching file: " + m.FileName);
 8             ftp.Get(LocalPath, m.FileName);
 9             if (DeleteRemoteFiles)
10             {
11                 Log.Debug("Deleting remote file: " + m.FileName);
12                 ftp.Delete(m.FileName);
13             }
14             result.AddTaskResult(string.Format(
15                 "<fileDownloaded>{0}</fileDownloaded>\n", m.FileName));
16         }
17     }
18     finally
19     {
20         ftp.Quit();
21     }
22 } 

Connecting to the FTP Server

The call to Connect in GetModifications above looks like this:

 1 public void Connect()
 2 {
 3     Log.Debug(String.Format("Connecting to remotehost: {0}...", RemoteHost));
 4     ftp.Connect();
 5     Log.Debug(String.Format("Connected, logging in with user: {0}", UserName));
 6     ftp.Login(UserName, Password);
 7     Log.Debug(String.Format("Switching to remote directory: {0}", RemoteDir));
 8     ftp.ChDir(RemoteDir);
 9 }

The only thing I haven't really covered here is the construction of the FTPClient member and the assignment of the its RemoteHost and Timeout properties which I think is pretty straightforward.

Conclusion

Writing plugins, whether for Source Control Providers or tasks (ITask), the process is pretty straightforward though there is little documentation on the subject. Be prepared to spend some time studying the other ISourceControl and ITask implementations included with CCNET and quite a bit of time debugging CCNET itself to understand how it works. One issue I've run into and that I've blogged about before which is when exceptions occur from within a Source Control Provider. In the current released version of CCNET (1.3) exceptions are not handled by the publishers for the project therefore you'll never get notified about connection problems etc. unless you use the log4net settings I mentioned.

The above implementation is lacking in a number of areas, like error checking, and is a very simplified example of code I'm using in production. I've also tweaked CCNET itself so that any exceptions raised within my FTP provider are handled by the build publisher so that I can get email when things fail. In fact, I've been working to get these, or similar, changes incorporated into CCNET.

[UPDATE: Feb 6, 2008] Fix image links

The other day I wrote about configuring CruiseControl.NET to send email when an exception occurred at the project level, meaning outside of the actual "build" which technically equates to the <tasks> block. I’ve since spent some time trying to understand why the CCNet was changed to remove support for PublishExceptions and found revision 2567 to IntegrationRunner.cs where it was removed with a comment of "all exceptions during build break the build
deprecating PublishExceptions attribute". That didn’t exactly provide the explanation I was looking for and AFAICT, this change has now left a gap in CruiseControl.NET where exceptions occurring outside of the <tasks> block can’t be published using a build publisher. So, unless you’re checking the log files regularly, or you’ve configured log4net using an SmtpAppender they’ll go unnoticed.

To figure this out I’ve been posting messages to the ccnet-devel group and thankfully I’m getting some response although it’s been a very slow conversation that’s had it’s share of missteps due, in part, to my incorrect phrasing of where exceptions were occurring. Originally, I was looking for guidance thinking I must be missing something. After a few more posts a change request and subsequent revision was made but the change addressed a situation I’m pretty sure can’t occur in the current code base short of a plug-in explicitly catching exceptions and setting the build status to "Exception" which isn’t what I was looking for.

I believe the necessary change to allow for integration exceptions to be published looks something like this (from IntegrationRunner.cs):

 1 public IIntegrationResult Integrate(IntegrationRequest request)
 2 {
 3     IIntegrationResult result = resultManager.StartNewIntegration(request);
 4     IIntegrationResult lastResult = resultManager.LastIntegrationResult;
 5     try
 6     {
 7         CreateDirectoryIfItDoesntExist(result.WorkingDirectory);
 8         CreateDirectoryIfItDoesntExist(result.ArtifactDirectory);
 9         result.MarkStartTime();
10         result.Modifications = GetModifications(lastResult, result);
11         if (result.ShouldRunBuild())
12         {
13             Log.Info("Building: " + request);
14             Build(result);
15             PostBuild(result
16             Log.Info(string.Format("Integration complete: {0} - {1}", 
17 result.Status, result.EndTime));
18         }
19         target.Activity = ProjectActivity.Sleeping;
20         return result;
21     }
22     catch (Exception ex)
23     {
24         result.Status = IntegrationStatus.Exception;
25         result.ExceptionResult = ex;
26         target.PublishResults(result);
27         return result;
28     }
29 }
30 

I’ve added a try...catch to handle exceptions that occur during the integration process and call PostBuild() which calls FinishIntegration() and PublishResults(). That way any exception occurring during the integration process can be handled by any of the available publishers including the email publisher. Btw, the old PublishExceptions code looked similar. Of course, the code presented here isn’t the only, nor perhaps even the best way to handle this situation personally, I’m not fond of PostBuild() appearing twice but adding a finally block seemed to unnecessarily complicate the code. Since I don’t understand the reasoning behind removing the PublishExceptions tag it’s hard to know what the desired behavior here is but I’m pretty sure what’s checked in is probably not what was intended.

There is one other change I think is important as well which is to modify IntegrationResult.ShouldRunBuild() as follows (from IntegrationResult.cs):

1 public bool ShouldRunBuild()
2 {
3     return status == IntegrationStatus.Unknown && 
4         (BuildCondition.ForceBuild == BuildCondition || HasModifications());
5 }  

This change rather subtle but it allows for an ISourceControl plug-in to handle it’s own exceptions and still prevent the build from executing by setting IntegrationStatus to something other than "Unknown". Without the check for IntegrationStatus.Unknown the build would run regardless of exceptions that occurred within the GetModifications() call. For example, here is a catch block you could use within your implementation of ISourceControl.GetModfications() to prevent the build from executing. However, this will only work if the above proposed changes are made to CCNet.

1 catch(Exception ex)
2 {
3     to.ExceptionResult = ex;
4     to.Status = ThoughtWorks.CruiseControl.Remote.IntegrationStatus.Exception;
5     return null;
6 } 

In the event of an exception you’ll have the opportunity to have the exception published using your configured publishers. Additionally, the web dashboard will reflect the fact that the build is left in an Exception state.

Anyway, I’m going continue to work this issue and see if I can get either the above changes or some other solution integrated into CCNet though the thread on the groups seems to have died.

[UPDATE: Jan 4, 2008] Here is an SVN patch for these changes.
[UPDATE: Jan 9, 2008] Fix for IntegrationRunner.Integrate to simply publish the results in the event of an exception.