Deploying Web Sites using TFS Deployer, PowerShell and FTP
I had a situation recently where I needed to deploy a web site into a production environment from TFS where the only way we could transfer files was via FTP. In case it's not obvious I didn't want to deploy every time the build ran, but rather, only when the build was declared good.
To do this, I used TFS Deployer (developed by Mitch, Chris, Darren and Geoff of Readify) available from CodePlex. TFS Deployer is a small utility that monitors changes in the build quality of a TFS Team Build and initiates a PowerShell script based on what the change in build quality was. It runs something like this:
A TeamBuild is executed and after completion the build has a quality of "Unexamined"
1. Someone runs some tests, checks the build is OK and decides the build quality can be progressed
2. TFS updates the build quality and raises an alert that the build quality has changed
3. On startup TFS Deployer registers itself as a listener for build quality events and picks up the alert. It examines the old/new qualities and determines if a PowerShell script should be run. If a script needs to be run then PowerShell is started and the script executed.
4. Results from the script are emailed to a specified email address of distribution list.
TFS Deployer Setup
I'll assume that TFS Deployer is installed and set up as per the instructions. To initiate the PowerShell script we set up a deployment mapping as follows:
<Mapping xmlns=""
Computer="MyBuildServer"
OriginalQuality="Unexamined"
NewQuality="Released to Production"
Script="DeployToProd.ps1"
NotificationAddress="[email protected]" />
That's about it. Pretty tough :-) Now for something a little harder.
Getting Ready for Deployment via FTP
OK, this is where it could get painful. PowerShell doesn't have any native FTP support. So the first option you have would be to use the native .NET FTP classes from but that's a real pain because you effectively have to implement your own FTP client in PowerShell. No thanks!
But don't despair - the open source Indy Project is a project that provides a wrapper and helper functions around for all the FTP calls you might want to make (plus a whole bunch of other low level networking goodness) and they have a .NET version of their library. Grab a copy of that (and the Mono.Security DLL that goes with it) and you'll have all you really need to get going.
I should also mention that because I'm deploying to a production box I'm assuming that IIS is already set up - after all it's very unlikely that we'd be dropping the IIS virtual directory on every deployment. We'll just be dropping our files in over the top of whatever is already there.
The PowerShell Script
Time for the script itself - I'll just take this section at a time. First up, the script is called from TFS Deployer which means we have access to the TFS build data for the build that just had it's quality changed. So we're going to set two variables for the folders where things live. $loc is the root of the build drop location, and $sourcefiles is where the published web site live (ie the stuff we need to deploy).
Set-Location $TfsDeployerBuildData.DropLocation;
$loc = get-location;
set-location "Mixed Platforms\Release\_PublishedWebSites\";
$sourceFiles = get-location;
Opening and Closing FTP Connections
Next we're going to create a few PowerShell script functions to support the opening and closing of our FTP connections.
When we open the connection we're first going to use the .NET assembly loader to bring in the Indy.Sockets library then use the methods in that to make the connection and then return the FTP connection object from the function. The close-method is a bit simpler in that we simply wrap the call to the FTP-disconnect. (yes, it's not really required as a function, but you never know when you might want to add logging, error handling, etc).
function Open-FTPConnection($ftphost, $username, $password) {
[void][Reflection.Assembly]::LoadFrom("C:\path\to\Indy.Sockets.dll")
$ftp = new-object Indy.Sockets.FTP
$ftp.Disconnect()
$ftp.Host = $ftphost
$ftp.Username = $username
$ftp.Password = $password
$ftp.Connect()
$ftp.Passive=$true;
return $ftp
}
function Close-FTPConnection($ftp) {
$ftp.Disconnect();
}Oh - For those not familiar with PowerShell syntax the :: operator lets us call static methods on a type and the [] wrap a type identifier.
FTP Miscellaneous Functions
Next we define a few more functions just to wrap up some of the basic FTP calls. Note: Download-FTPFile isn't used - it's just there for your reference.
function Get-FTPCurrentLocation($ftp) {
return $ftp.RetrieveCurrentDir();
}
function Download-FTPFile($ftp, $sourceFileName, $targetDir) {
$ftp.Get($sourceFileName, ($targetDir + $sourceFileName), $true, $false);
}
function Upload-FTPFile($ftp, $sourceFileName, $targetFileName) {
$ftp.Put($sourceFileName, $targetFileName, $false);
}Nothing overly complex in that.
Getting the Contents of an FTP Folder
Now we get to something a bit more interesting. Here we're creating a function that iterates over the contents of the FTP location and optionally deleting files as it goes. It will return a list of sub-folders in the folder for later use.
The function works as follows:
1. Get the directory listing for the current FTP location.
2. Checks if the current directory is the root folder or not - appends a trailing slash if it isn't.
3. For each item in the folder...
3a. Parses the string to get the file name
3b. Checks if the item is a directory or a file (directories have a "d" in their attributes)
3c. For folders we return the full path to the sub-folder.
3d. For files we check the delete flag and nuke the file if it is set.
You'll notice that the $result.add($name) call is cast to a [void]. If we don't do this then when we return from the function we get 2 sets of files. One from the $result object and one for each file name that was written to the output stream by the $result.add() method call.
function Get-FTPFolders($ftp, [bool]$removeFiles) {
$ls = new-object System.Collections.Specialized.StringCollection;
$result = new-object System.Collections.Specialized.StringCollection;
$ftp.List($ls, "", $true);
$currdir = Get-FTPCurrentLocation($ftp);
if($currdir -ne "/") {
$currdir = $currdir + "/";
}
foreach ($item in $ls)
{
[string[]]$fields=[Regex]::Split($item, " +");
$startField=8; #the file/directory name starts after 8 fields
[string]$name=$currdir;
#make sure we join up file names that were split (ie ones with spaces)
for ($field=$startField; $field -lt $fields.Length; $field++)
{
if ($field -eq $startField)
{
$temp = ""
} else
{
$temp = " "
}
$name += $temp + $fields[$field];
}
if ($item.StartsWith("d"))
{ #directory
[void]$result.Add($name);
}
else
{
if ($item.StartsWith("-")) { #files have '-' as first character
if ($removeFiles)
{
$ftp.Delete($name);
}
}
}
}
return $result
}Iterating/Deleting the FTP Folder Structure
Next I have two more methods. One to read through the contents of the FTP folder hierarchy, and one to clear it out. Both methods are roughly the same with only variations for the call to Get-FTPFolders.
function Get-FTPTree($ftp)
{
$thisdir = Get-FTPCurrentLocation($ftp);
$thisdir;
$subfolders = (Get-FTPFolders $ftp $false);
if ($subfolders -ne $null) {
foreach ($xitem in $subfolders)
{
$ftp.ChangeDir($xitem);
Get-FTPTree($ftp);
}
}
$ftp.ChangeDir($thisdir);
return;
}
function Clean-FTPTree($ftp)
{
$thisdir = Get-FTPCurrentLocation($ftp);
$thisdir;
$subfolders = (Get-FTPFolders $ftp $true);
if ($subfolders -ne $null) {
foreach ($xitem in $subfolders)
{
$ftp.ChangeDir($xitem);
Clean-FTPTree($ftp);
$foldername = $xitem.split("/");
$ftp.ChangeDir($thisdir);
$ftp.RemoveDir($foldername[$foldername.Count - 1]);
}
}
$ftp.ChangeDir($thisdir);
return;
}
Note that in the Clean method we delete folders by stripping the folder name off the end of the full path, stepping back up a level and then calling the removedir method. It's not pretty, but it works.
Putting it all Together
Now we have everything in place let's actually do what we need to do. Here's what happens:
1. We make the connection to the FTP server.
2. We clean out the existing FTP tree
3. We get a list of all the files we're going to upload.
4. We copy each file individually to the ftp server, creating folders where required (note the filename processing).
5. Just to make sure things are right - we get the contents of the site so we can check the upload worked.
6. Close the connection - we're done!
#Make a connection
$f = Open-FTPConnection "my.ftp.server.com" "ftp_user_account" "ftp_account_password";
write-output "----CLEANING----";
Clean-FTPTree($f);
write-output "----UPLOADING----";
$localfiles = (get-childitem $sourcefiles -r)
foreach ($localfile in $localfiles) {
$remfilename = $localfile.FullName.Replace($sourcefiles.ProviderPath, "");
$remfilename = $remfilename.Replace("\", "/");
if ($localfile.Attributes -eq "Directory") {
Write-Output ("Creating " + $localfile.FullName + ":" + $remfilename);
$f.MakeDir($remfilename);
}
else {
Write-Output (" Uploading " + $localfile.FullName + ":" + $remfilename);
upload-ftpfile $f $localfile.FullName $remfilename
$remfilename;
}
}
write-output "----VERIFYING----";
Get-FTPTree($f);
Close-FTPConnection $f
One thing you might notice is that we use the ProviderPath property of the $sourcefiles variable. This is because $sourcefiles is a PathInfo object and when TFS Deployer runs the PathInfo will contain a UNC path pointing to a network share - if we just use the Path property then we will get the PowerShell Provider identifier in the string causing the FTP upload to fail.
Hopefully this is a good starting point for you if you are trying to do the same thing.