You're deploying it wrong! TeamCity, Subversion & Web Deploy part 1: Config transforms

If you publish a web application using CTRL-C and CTRL-V, you’re deploying it wrong.

If you manually run an Xcopy command, you’re deploying it wrong.

If you use an FTP client to move your files to a remote server, you’re deploying it wrong.

If not everyone is following exactly the same release process, you’re deploying it wrong.

If publishing involves any manual handling of Web.config, you’re deploying it wrong.

This might seem a little sensationalist but after Scott Hanselman set a high “doing it wrong” bar in his excellent Web Deployment Made Awesome: If You're Using XCopy, You're Doing It Wrong video, he made it pretty clear just how wrong many people have been doing deployment.

To be fair, there really haven’t been a lot of alternatives. Some of the “wrong” scenarios above have only been satisfactorily addressed in the latest generation of tools. But now that they have, the excuses for “deploying it wrong” are running out.

Predictability, predictability, predictability

What’s so wrong with all these practices? They’re unpredictable and in the world of configuring and releasing applications, predictability is key. Consider the opportunities for things to go wrong under the scenarios described above:

1. Development environment connection strings or other app settings are accidentally published to production.
2. Network connectivity drops out part way through a deployment leaving half the content in version 1 and the other half in version 2.
3. A critical flaw is found in the release and you need to rapidly roll back to the previous version.
4. An urgent release is required but the one person who understands the process (and possibly the credentials), is not contactable.

There are actually two different issues here – configuration management and deployment, the latter being a question of manual versus automated. I want to look at how we can tackle this through a combination of config transforms, website packaging, Web Deploy and finally, automation via TeamCity to both development and production environments. Throughout all this I’m going to be working towards the principle of least surprise because in the world of deployments, surprise is rarely a good thing.

5 easy parts

This process is not revolutionary, even though it’s mostly fairly new practices courtesy of the current generation of .NET tooling. However, what I’m going to do a bit differently in this post is to try and make this really, really easy to understand. I’m going to write what I wish I could have found when I embarked on this process because quite frankly, I think there’s an unmet need.

Many of the concepts I’ll discuss only make sense when you understand them in the order in which I’ve presented them. Subsequently, I’m going to break this down into 5 sequential parts and so long as you understand them in sequence, the whole thing should make sense. This might seem a bit verbose but there’s a lot to this subject and I’m going to illustrate as much as possible so there’s no uncertainty. Here’s how it’s going to look:

1. Config transforms
2. MSBuild and deployable packages
3. Publishing with Web Deploy
4. Continuous builds with TeamCity
5. Web Deploy with TeamCity

I’m also only going to talk about web apps and not touch on databases at this stage. Yes, databases are an important part of deployment but there are a number of reasons why we need to treat them very differently. For example, managing change, particularly when releasing to a production environment, is quite a different deployment proposition. Another post for another day.

Config transforms

One of the challenges you’re always going to have with applications which migrate through the usual lifecycle of environments such as development, test and production is getting the configuration context right. For example, your database connection string will probably (hopefully) indicate a different server and the password of the account you connect with will be different (again, hopefully). Similarly, you may want to turn off custom errors and enable tracing in your development environment but that’s certainly not the state you want your app to persist in when it hits production.

I’ve seen a lot of different practices to tackle this in the past, probably the worst of which is attempting to publish apps without the Web.config in deference to trusting the one in the target environment is probably still correct. Or alternatively, publishing the app but then hand editing the Web.config in the target. Both these practices are extremely fragile and if you manage to consistently deploy apps and get this right every time it’s more likely good luck than good management.

Another approach is the programmatic switching of configurations depending on the environment. This might be hard coded at the point where a connection string is referenced (if server = “Dev” then “devDb” else “prodDb”) or there may be multiple connection strings defined in the Web.config then switched in the same location as the previous example (if server = “Dev” then “devConnString” else “prodConnString”). The problem with both these is not only do you start to build up a bunch of fragile conditional logic (what happens if the definition of “Dev” – such as the server name – changes?), it’s a bit, well, “non-standard”. It’s a bad thing in terms of it being a very bespoke implementation for something that really should be a pretty consistent practice across all applications.

I’m sure there are many other ways people have managed their configurations but let’s move on to config transforms. Essentially this is just a means of explaining the substitution of Web.config settings based on the build configuration. By default, you’ll have a “Debug” and a “Release” build configuration in your project. Because this exercise is ultimately about deploying into another environment I’m going to add a custom build configuration in my test solution (just a standard ASP.NET we app called “AutoDeployProject”) by first going to the Configuration Manager:

Selecting a “New…” configuration:

Then creating a configuration called “Deploy-Dev” which will just be a copy of the existing “Debug” configuration:

Now we can move on to creating the transforms. As of Visual Studio 2010, we now have an “Add Config Transforms” context menu on the Web.config:

What this will do is produce a new config file for each build configuration in the app. Normally you'll have “Debug” and “Release” build configurations which will produce a Web.Debug.config and a Web.Release.config, both neatly cascaded beneath the original Web.config, but of course we just added a third build configuration so here’s what we’ll see:

If you take a look inside, the release version, you’ll see something similar to this:

<?xml version="1.0"?>
<configuration
  xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
  <system.web>
    <compilation xdt:Transform="RemoveAttributes(debug)" />
  </system.web>
</configuration>

Explained simply, when we’re in release mode we really don’t want to enable debug and by removing the attribute altogether the app will apply the default value which is “false”. Strangely though, the contents of our Web.Deploy-Dev.config file is identical to the “Release” version even though it’s based on the “Debug” build configuration. We’ll go in there and remove the <compilation> node because for this example, we want debug enabled when it lands on our development server.

There are many, many other possible transforms we can apply including replacing, inserting and deleting nodes or replacing and deleting individual attributes. There’s a great MSDN article about Web Deployment: Web.Config Transformation which goes into a lot more detail.

One thing I should mention now is that I cheated a little bit in the example above. All new Visual Studio 2010 web projects already have these transform files in place. If you delete them (as I did) or have an earlier generation project, that was how you add them in manually.

Let’s now create a couple of typical configurations in the Web.config which we can transform later on when we publish. We’ll start with a connection string that uses a data source on the local machine:

<connectionStrings>
  <add name="AutoDeployDb" connectionString="Data Source=(local);Initial
    Catalog=AutoDeploy;User ID=AutoDeployUser;Password=Passw0rd"/>
</connectionStrings>

And then a custom app setting which in this case is going to be the settings for the Chart control HTTP Handler. This is a good example as it must be an absolute path on the file system so it’s the sort of thing that easily change between environments (and it’s frustrated me greatly in the past!):

<appSettings>
  <add key="ChartImageHandler"
    value="storage=file;timeout=20;dir=c:\TempImageFiles\;" />
</appSettings>

What we’re trying to do here is if get the Web.config into a state that’s perfect for local development. All of these settings are what I want when I’m running the app on my local machine. The transforms only get applied when we publish so the raw state of the Web.config must be something that can run happily throughout the development process on my machine.

Now let’s fill in the Web.Deploy-Dev.config file with some values which are suitable for the development server. We’ll give it a whole new connection string with a new data source and password and set the chart image handler to use a different directory. Here’s what the entire file will look like:

<?xml version="1.0"?>
<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-
  Transform">
  <connectionStrings>
    <add xdt:Transform="SetAttributes" xdt:Locator="Match(name)" 
      name="AutoDeployDb" connectionString="Data Source=MyDevServer;Initial 
      Catalog=AutoDeploy;User ID=AutoDeployUser;Password=s*#@Kdsl" />
  </connectionStrings>
  <appSettings>
    <add xdt:Transform="Replace" xdt:Locator="Match(key)" 
      key="ChartImageHandler"
      value="storage=file;timeout=20;dir=d:\AutoDeploy\TempImageFiles\"/>
  </appSettings>
</configuration>

And that’s it – that’s all that’s involved in a config transform. But will it work?

Testing the transforms

Now that we’ve got all that configured we really want to test it works. One thing I’ve learnt from much trial and error with software is that if you don’t individually test each discrete configuration in an app, your troubleshooting quickly becomes very messy and very painful. The easiest way to test the config transforms is to first select the “Deploy-Dev” build from the build configuration dropdown on the toolbar then right click on the web app in the solution explorer and choose “Publish…”:

When we do this for the first time, we’ll get a default profile name of “Profile1”. Let’s rename this to “Dev” and set the publish method to “File System” and the target location to somewhere temporary on the machine (we’ll change all this later). Here’s how it should look:

Now we’ll navigate over to the target location in Windows Explorer and check the contents of the Web.config file. If the connection string uses the data source name and the password we entered in the config transform and the chart image handler value has also been updated accordingly then the config transform has been successful:

So that’s part 1 done successfully. One important point to note here is that CTRL-C and CTRL-V or Xcopy would never work in this case because it won’t apply the transforms. You must follow a process that’s aware of this concept and applies it correctly which is obviously what happens using the publish feature of Visual Studio.

Coming next…

Config transforms are very basic but they solve a complex problem which has been the cause of much ire for many people over the years. Next up in the automation process we’ll get the application built and packaged via the command line.