A Real World CruiseControl.NET Setup
I've been using CCNet with teams for many years now and it works beautifully. Of course these days you can also do continuous integration with a number of other products like Microsoft's Team Foundation Server or any of the various other offerings out there, but CCNet is the still going strong with version 1.2 recently released.
The details below are for the configuration I've used on my most recent project (names changed for privacy). It's a line of business web application consisting of around 25 projects and a few hundred web forms.
The Environment
The development environment has a 4 CPU HP server running VMWare ESX and has a number of test servers and so forth setup on it. The build server itself is one of the virtual servers. It is configured for 2 CPU's, 1 GB Ram and 20 GB of disk.
For simplicity, this server is also where the Subversion source repository is located.
The disk space is split into 3 drives. The system disk has 8 GB on it, there is a 4 GB disk for build log files, and the rest is allocated to the build server and the subversion repository.
Developers have the TortoiseSVN client on their machines for Subversion access and the CCTray tool installed so they can see the current state of the build server (and know when someone breaks the build).
Gemini is used as the issue tracking system and we have integrated Subversion commits into the Gemini solution.
Server Software
The build server has the following software installed:
- CruiseControl.NET (obviously!)
- NUnit 2.2
- FxCop 1.35
- NAnt 0.85 (and NAnt-Contrib 0.85)
- NCover
- NCoverExplorer
- Microsoft .NET Framework SDK
- SourceMonitor
- IIS (for the CCNet WebDashboard interface)
You'll notice that Visual Studio is NOT installed on the build server. Also, the software being built uses 3rd party components from Telerik, Dundas and others. These 3rd party products are also NOT installed on the build server.
Finally, we use MSBuild in the process. You'll need the MSBuild Logger in order to get MSBuild output readable in the Web Dashboard.
The Build Process
The build process is pretty simple. Here's how it works:
- Developer commits code into Subversion
- CruiseControl.NET detects that a commit has occurred
- The buils direcortory is cleaned
- CCNet checks out the code into the local working directory on the build server
- 3rd party DLL's are copied into the bin directory for the web site
- .config.template files are copied to .config files
- Assembly numbers are updated to match the Subversion revision number
- MSBuild is called to compile the solution
- Unit tests are run (with NCover for code coverage)
- NCoverExplorer is run over the NCover output
- FxCop is run. This step can fail, as we expect some FxCop validation errors on legacy code
- SourceMonitor is run
If any step (except FxCop) fails, we stop immediately.
If it all works, we then create a folder containing the application in a ready-to-release state as follows:
- Clean out the existing contents in the release folder
- Copy everything from PrecompiledWeb
- Create default blank folders (e.g. upload folders, etc that are part of the applications standard configuration)
- Copy in extra non-build items, including things such as deployment untilities and help files
And that's it.
One interesting thing to note is what we do with .config files. In VS2005 standard web applications require a web.config file to help figure out where references are, and so forth. Unfortunately in a development environment developers often use different config files to those of other developers and those on the build server (eg connection strings, logging and debug settings, etc). We use the concept of a baseline web.config file called web.config.template. This .config.template file is what is used by the build server to set the defaults for the application both in terms of compilation and standard deployment/runtime configuration options.
The Web Dashboard
The web dashboard is the web interface to the ccnet build server. We have configured this to show more information about the build process than that provided in the base CruiseControl.NET installation.
Our dashboard looks like this:
The build log now included the Subversion revision number and provides people with the ability to view the NCover output using NCoverExplorer, the MSBuild output (for finding why builds fail - as you can see, sometimes they do!) and SourceMonitor output.
How Is It Configured?
All the configuration information here assumes you've already set up CruiseControl.NET and understand the basics of how to configure it.
Let's start with the web dashboard. We've changed the <buildPlugins> section in dashboard.config to look like this:
P.S. Apologies for the formatting - blogger got all "helpful" and removed the leading whitespace
<buildPlugins>
<buildReportBuildPlugin>
<xslFileNames>
<xslFile>xsl\header_with_revision.xsl</xslFile>
<xslFile>xsl\modifications.xsl</xslFile>
<xslFile>xsl\compile.xsl</xslFile>
<xslFile>xsl\unittests.xsl</xslFile>
<xslFile>xsl\MsTestSummary.xsl</xslFile>
<xslFile>xsl\NCoverExplorerSummary.xsl</xslFile>
<xslFile>xsl\fxcop_new_summary.xsl</xslFile>
<xslFile>xsl\sourcemonitor-summary.xsl</xslFile>
<xslFile>xsl\compile-msbuild.xsl</xslFile>
</xslFileNames>
</buildReportBuildPlugin>
<buildLogBuildPlugin />
<xslReportBuildPlugin description="NUnit Details" actionName="NUnitDetailsBuildReport" xslFileName="xsl\tests.xsl" />
<xslReportBuildPlugin description="NUnit Timings" actionName="NUnitTimingsBuildReport" xslFileName="xsl\timing.xsl" />
<xslReportBuildPlugin description="NCover Report" actionName="NCoverBuildReport" xslFileName="xsl\NCoverExplorer.xsl" />
<xslReportBuildPlugin description="NCover Details" actionName="NCoverDetailsReport" xslFileName="xsl\NCover-details.xsl" />
<xslReportBuildPlugin description="FxCop Report" actionName="FxCopBuildReport" xslFileName="xsl\FxCop_new.xsl" />
<xslReportBuildPlugin description="MSBuild Report" actionName="MSBuildOutputBuildPlugin" xslFileName="xsl\msbuild.xsl"/>
<xslReportBuildPlugin description="NAnt Output" actionName="NAntOutputBuildReport" xslFileName="xsl\Nant.xsl" />
<xslReportBuildPlugin description="NAnt Timings" actionName="NAntTimingsBuildReport" xslFileName="xsl\NantTiming.xsl" />
<xslReportBuildPlugin description="SourceMonitor Top 30" actionName="SourcemMonitorTop15BuildReport" xslFileName="xsl\SourceMonitor-Top15.xsl"/>
<xslReportBuildPlugin description="SourceMonitor Files" actionName="SourcemMonitorFileBuildReport" xslFileName="xsl\sourcemonitor-group-by-file.xsl"/>
<xslReportBuildPlugin description="SourceMonitor Metrics" actionName="SourcemMonitorMetricBuildReport" xslFileName="xsl\sourcemonitor-group-by-metric.xsl"/>
There is a new .xsl file to add the subversion revision to the build output. See my previous post for more information.
We also add a few new build plugins to allow us to view the XML output from programs like SourceMonitor & NCover in the web dashboard.
The Project's CCNet Configuration
When CCNet is set up you need to edit ccnet.config to tell it about the projects you are building.
The configuration for our project is as follows
<project name="xxxx">
<workingDirectory>D:\CCProjects\xxxx</workingDirectory>
<artifactDirectory>E:\CCArtifacts\xxxx</artifactDirectory>
<webURL>http://ccnet.server.local/ccnet</webURL>
<modificationDelaySeconds>0</modificationDelaySeconds>
<publishExceptions>true</publishExceptions>
<triggers>
<intervalTrigger seconds="60" />
</triggers>
<sourcecontrol type="svn">
<trunkUrl>"file:///D:/svnrepos/xxxx/trunk"</trunkUrl>
<workingDirectory>D:\CCProjects\xxxx</workingDirectory>
<executable>C:\Program Files\Subversion\bin\svn.exe</executable>
</sourcecontrol>
<tasks>
<nant>
<executable>C:\program files\nant-0.85-rc4\bin\nant.exe</executable>
<baseDirectory>D:\CCProjects\xxxx</baseDirectory>
<nologo>true</nologo>
<buildFile>default.build</buildFile>
<logger>NAnt.Core.XmlLogger</logger>
<targetList>
<target>compile</target>
</targetList>
<buildTimeoutSeconds>6000</buildTimeoutSeconds>
</nant>
</tasks>
<publishers>
<merge>
<files>
<file>D:\CCProjects\xxxx\_msbuild.xml</file>
<file>D:\CCProjects\xxxx\_nunit_*.xml</file>
<file>D:\CCProjects\xxxx\_ncover_*.xml</file>
<file>D:\CCProjects\xxxx\_coverageReport.xml</file>
<file>D:\CCProjects\xxxx\_fxcop_*.xml</file>
<file>D:\CCProjects\xxxx\_sm_*.xml</file>
</files>
</merge>
<xmllogger />
</publishers>
</project>
You can see that the sourcecontrol section is set up for Subversion (type="svn") and that we use an NAnt task to kick off the build when a commit is detected in Subversion.
The <publishers> section is where all the xml output files are processed and merged to produce the information displayed on the web dashboard. As a naming convention all XML build logs are named as _file.xml. This makes it easier to clean up things when the next build starts.
The actual NAnt task that is executed when a change is detected is shown here
<?xml version="1.0"?>
<project name="FullSolution" default="compile">
<property name="contrib" value="C:\program files\nantcontrib-0.85-rc4" overwrite="true" />
<property name="sourceDir" value="D:\CCProjects\xxxx" overwrite="true" />
<property name="svnUri" value="file:///D:/svnrepos/xxxx/trunk" overwrite="true" />
<property name="winDir" value="c:\windows" overwrite="true" />
<property name="nunit-location" value="c:\Program Files\NUnit-Net-2.0 2.2.8\bin" overwrite="true" />
<property name="nunit-console" value="${nunit-location}\nunit-console.exe" overwrite="true" />
<property name="fxcop-console" value="c:\Program Files\Microsoft FxCop 1.35\FxCopCmd.exe" overwrite="true" />
<property name="ncover-console" value="c:\Program Files\ncover\ncover.console.exe" overwrite="true" />
<property name="ncoverexplorer-console" value="C:\Program Files\NCover\NCoverExplorer\ncoverexplorer.console.exe" overwrite="true" />
<target name="*">
<nant target="${target::get-current-target()}" buildfile="xxxx/default.build" />
<nant target="releaseIt" buildfile="makeRelease.build" />
</target>
</project>
First we just set up a number of properties pointing to the location of the various tools we'll use when the build actually executes.
Next we call two NAnt targets, one that does to build, and one that does the release. If the build step fails then the release step is skipped.
The NAnt Build Script
Now, finally we get to the fun part. The NAnt build script.
There's a fair bit of detail in the build, but the targets are as follows:
SVN: Gets the source from Subversion and the revision number
Compile: Calls clean and SVN targets first, then set's the Assembly version numbers, checks codebehind vs codefile issues and executes MSBuild before running post build targets
Clean: Removes previous compilation output and xml log files
GetAssemblyVersion: Reads the assembly version number from AssemblyInfo.cs and stores is in NAnt properties
SetAssemblyVersion: Edits the AssemblyInfo.cs files for various projects using the Subversion revision number
CodeBehindToFile: Switches any "CodeBehind" ASP directives to "CodeFile" directives. Developers typically use the Web Application project in VS2005 so that local compiles are quick. The build server uses the Web Site project so that we get full compilation up front and avoid any nasty run time surprises. Unfortunately web applications use CodeBehind in the aspx pages, while the web site uses the CodeFile directive. This ensures that developers can use either without worrying about screwing up the build server.
unitTests: It runs the unit tests! But you'll note it does it via NCover so that we can get coverage analysis performed.
nCoverExplorer: It processes the NCover XML output using the console application and creates summarised XML output that makes it a lot easier to view results in the Web Dashboard.
FxCop: Pretty simple - just runs FxCop over all the .DLL's it can find. We skip 3rd party .DLL's as we can't really do anything about them if they have FxCop violations.
SourceMonitor: Runs the sourceMonitor console app to process the code and calculate cyclomatic complexity values and finds the nastiest, ugliest code it can so that we have a hot-list of methods that need refactoring.
Here's the NAnt build script in full (well, slightly edited for privacy):
<?xml version="1.0" ?>
<project name="xxxx" default="compile" basedir=".">
<description>Build the xxxx Application</description>
<property name="clean.pattern.bin" value="**/bin/**/*" />
<property name="clean.pattern.obj" value="**/obj/**/*" />
<property name="clean.pattern.precompiled" value="**/precompiledweb/**/*" />
<property name="clean.pattern.assemblyinfo" value="**/assemblyinfo.cs" />
<property name="clean.pattern.xml" value="_*.xml" />
<property name="build.major" value="1"/>
<property name="build.minor" value="0"/>
<property name="build.build" value="0"/>
<property name="build.revision" value="0"/>
<property name="svn.revision" value="0"/>
<property name="assemblyinfo.location" value="xxxx\App_Code\AssemblyInfo.cs"/>
<property name="counter" value="0"/>
<property name="codemetrics.output.dir" value="c:\temp"/>
<property name="sourcemonitor.executable" value="c:\Program Files\SourceMonitor\SourceMonitor.exe"/>
<target name="svn" description="Get source code from Subversion">
<loadtasks assembly="${contrib}\bin\NAnt.Contrib.Tasks.dll" />
<echo message="loaded tasks from ${contrib}\bin\NAnt.Contrib.Tasks.dll" />
<svn-update destination="${sourceDir}" uri="${svnUri}"/>
<echo message="Retrieving Subversion revision number"/>
<exec
program="svn.exe"
commandline='log "${project::get-base-directory()}" --xml --limit 1'
output="_revision.xml"
failonerror="false"/>
<xmlpeek
file="_revision.xml"
xpath="/log/logentry/@revision"
property="svn.revision"
failonerror="false"/>
<echo message="Using Subversion revision number: ${svn.revision}"/>
</target>
<target name="compile" depends="clean,svn">
<copy file="ThirdPartyAssemblies\RadAjax.Net2.DLL" tofile="xxxx/bin/RadAjax.Net2.DLL" overwrite="true"/>
<copy file="xxxx\xxxx.config.template" tofile="xxxx\xxxx.config" overwrite="true"/>
<call target="SetAssemblyVersion"/>
<call target="CodeBehindToFile"/>
<exec
program="${winDir}\microsoft.net\framework\v2.0.50727\msbuild.exe"
commandline='xxxx.sln /p:Configuration=Release /logger:ThoughtWorks.CruiseControl.MsBuild.XmlLogger,..\ThoughtWorks.CruiseControl.MSBuild.dll /noconsolelogger /nologo /noautorsp'
output="_msbuild.xml"
failonerror="true"/>
<call target="unitTests"/>
<call target="ncoverExplorer"/>
<call target="FxCop"/>
<call target="sourcemonitor"/>
</target>
<target name="unitTests">
<foreach item="File" property="filename">
<in>
<items>
<include name="xxxx\bin\*.dll"></include>
<include name="**\bin\*\*tests.dll"></include>
<exclude name="**\Castle.DynamicProxy.dll" />
</items>
</in>
<do>
<exec program="${ncover-console}" workingdir="${path::get-directory-name(filename)}" commandline=""${nunit-console}" ${filename} /xml:${project::get-base-directory()}\_nunit_${path::get-file-name-without-extension(filename)}.xml /nologo //x ${project::get-base-directory()}\_ncover_${path::get-file-name-without-extension(filename)}.xml" failonerror="true"/>
</do>
</foreach>
</target>
<!-- Run NCover Explorer over results for summary output -->
<target name="ncoverExplorer">
<echo message="Starting NCoverExplorer report generation..."/>
<exec program="${ncoverexplorer-console}"
workingdir="${project::get-base-directory()}" >
<arg value="_ncover_*.xml" />
<arg value="/r:5"/>
<arg value="/x:_CoverageReport.xml" />
<arg value="/e" />
<arg value="/p:xxxx" />
<arg value="/m:85" />
</exec>
</target>
<target name="clean" description="remove generated files">
<delete>
<fileset>
<include name="${clean.pattern.obj}"/>
<include name="${clean.pattern.bin}"/>
<include name="${clean.pattern.precompiled}"/>
<include name="${clean.pattern.assemblyinfo}"/>
<include name="${clean.pattern.xml}"/>
</fileset>
</delete>
</target>
<!-- Populates variables (build.major, build.minor, build.build, and build.revision) with values
from AssemblyInfo.cs. If property assemblyinfo.location is undefined, it will attempt to
read AssemblyInfo.cs from the current directory. -->
<target name="GetAssemblyVersion" description="Populates variables with the current version." >
<script language="C#">
<code><![CDATA[
public static void ScriptMain(Project project) {
string fileName = Path.Combine(project.BaseDirectory, project.Properties["assemblyinfo.location"]);
StreamReader reader = new StreamReader(fileName);
{
Regex expression = new Regex(@"AssemblyVersion");
string line = reader.ReadLine();
while (line != null) {
Match match = expression.Match(line);
if (match.Success) {
Regex pattern = new Regex("[0-9]+");
MatchCollection matches = pattern.Matches(line);
if (matches.Count != 4)
throw new Exception(string.Format("Version number in {0} has incorrect format.", fileName));
int major = int.Parse(matches[0].Value);
int minor = int.Parse(matches[1].Value);
int build = int.Parse(matches[2].Value);
int revision = int.Parse(matches[3].Value);
project.Properties["build.major"] = major.ToString();
project.Properties["build.minor"] = minor.ToString();
project.Properties["build.build"] = build.ToString();
project.Properties["build.revision"] = revision.ToString();
break;
}
line = reader.ReadLine();
}
}
reader.Close();
}
]]></code>
</script>
<echo message="major: ${build.major}"/>
<echo message="minor: ${build.minor}"/>
<echo message="build: ${build.build}"/>
<echo message="revision: ${build.revision}"/>
</target>
<!-- Change CodeBehind=" instances to CodeFile=" -->
<target name="CodeBehindToFile" description="Fixes the web application web site annoyance">
<foreach item="File" property="filename">
<in>
<items>
<include name="xxxx\*.aspx"></include>
<include name="xxxx\UserControl\*.ascx"></include>
</items>
</in>
<do>
<script language="C#">
<code><![CDATA[
public static void ScriptMain(Project project) {
string contents;
using ( StreamReader reader = new StreamReader ( project.Properties [ "filename" ] ) )
{
contents = reader.ReadToEnd ();
reader.Close ();
}
contents = System.Text.RegularExpressions.Regex.Replace(contents,"CodeBehind=\"","CodeFile=\"",System.Text.RegularExpressions.RegexOptions.IgnoreCase);
using ( StreamWriter writer = new StreamWriter ( project.Properties [ "filename" ] , false ) )
{
writer.Write ( contents );
writer.Close ();
}
}
]]></code>
</script>
</do>
</foreach>
</target>
<target name="SetAssemblyVersion" description="Increments/Sets the AssemblyVersion value" depends="GetAssemblyVersion">
<!-- Increment the build.revision value -->
<foreach item="File" property="filename">
<in>
<items>
<include name="xxxx\App_Code\AssemblyInfo.cs"></include>
<include name="Project2\AssemblyInfo.cs"></include>
<include name="Project3\AssemblyInfo.cs"></include>
</items>
</in>
<do>
<script language="C#">
<code><![CDATA[
public static void ScriptMain(Project project) {
string contents = "";
StreamReader reader = new StreamReader(project.Properties["filename"]);
contents = reader.ReadToEnd();
reader.Close();
string replacement = string.Format(
"AssemblyVersion(\"{0}.{1}.{2}.{3}\")]",
project.Properties["build.major"],
project.Properties["build.minor"],
project.Properties["build.build"],
project.Properties["svn.revision"]
);
string newText = Regex.Replace(contents, @"AssemblyVersion\("".*""\)\]", replacement);
StreamWriter writer = new StreamWriter(project.Properties["filename"], false);
writer.Write(newText);
writer.Close();
}
]]></code>
</script>
</do>
</foreach>
</target>
<target name="FxCop">
<exec program="${fxcop-console}" commandline="/file:precompiledweb/xxxx/bin /out:_fxcop_all.xml /directory:"${nunit-location}" /searchgac" failonerror="false"/>
<!--
<foreach item="File" property="filename">
<in>
<items>
<include name="precompiledweb\xxxx\bin\*.dll"></include>
<exclude name="**\Castle.DynamicProxy.dll" />
</items>
</in>
<do>
<exec program="${fxcop-console}" commandline="/file:${filename} /out:_fxcop_${path::get-file-name-without-extension(filename)}.xml /directory:"${nunit-location}" /searchgac" failonerror="false"/>
</do>
</foreach>
-->
</target>
<target name="sourcemonitor">
<!-- Create input command file -->
<property name="sourcemonitor.input" value="${codemetrics.output.dir}\sm_xxxx_cmd.xml" />
<echo file="${sourcemonitor.input}" append="false" failonerror="false">
<![CDATA[
<?xml version="1.0" encoding="UTF-8" ?>
<sourcemonitor_commands>
<write_log>true</write_log>
<command>
<project_file>${codemetrics.output.dir}\sm_xxxx.smp</project_file>
<project_language>CSharp</project_language>
<source_directory>${project::get-base-directory()}</source_directory>
<include_subdirectories>true</include_subdirectories>
<checkpoint_name>${svn.revision}</checkpoint_name>
<export>
<export_file>${project::get-base-directory()}\_sm_summary.xml</export_file>
<export_type>1</export_type>
</export>
</command>
<command>
<project_file>${codemetrics.output.dir}\sm_xxxx.smp</project_file>
<checkpoint_name>${svn.revision}</checkpoint_name>
<export>
<export_file>${project::get-base-directory()}\_sm_details.xml</export_file>
<export_type>2</export_type>
</export>
</command>
</sourcemonitor_commands>
]]>
</echo>
<!-- Execute the commands -->
<exec program="${sourcemonitor.executable}" commandline="/C ${sourcemonitor.input}" failonerror="false" />
<style style="SourceMonitor-Top15Generation.xsl" in="${project::get-base-directory()}\_sm_details.xml" out="${project::get-base-directory()}\_sm_top15.xml" />
<delete file="${codemetrics.output.dir}\sm_xxxx.smp" failonerror="false" />
</target>
</project>
Hopefully you find that useful, and that there's some ideas in there you can borrow. If you've got any questions please feel free to ask.