On Building Deployments From Source Control

by mgordon 29. April 2008 08:13

Recently, I needed to make some changes to an older application that was created before I joined the team.  I set out to find the most recent version of the source for this application.  The team, at the time the application was built, used a combination of Visual Source Safe and backing up copies of the source (not the source database) to both a share on the network and also an external hard drive that was hanging off one of the developers' workstations.  I first checked the backups on the hard drive and found 20 copies of the source there.  Apparently, most all the copies had been moved at the same time because the time stamps on the files were identical.  Some of the folders had dates in the names, but had sub-folders with multiple versions in them.  In checking the share, I found basically the same thing.  Next, I checked VSS and found the source checked out by an employee no longer with the company and that about 5 months of bug fixes had been made without checking the code back in.

So, after the shiver went down my spine, I went about trying to decide how I was going to handle the fact that I had a version running in production that was about 5 months newer than the latest version of source code I could find.  Not a pretty situation...

This experience just underscores the value of having and using an automated build system and a process that keeps it a central part of the overall approach to things.  With a build server in place, the release versions are generated FROM source control.  So, unless you lose your repository, it's impossible to deploy a version that isn't in the source database.

I found another problem with the source, as well.  It depended on several third party controls.  Apparently, the controls had been installed on the developer's computer and references from the project to the third party assemblies had been made to the installed location.  What this meant for me was that after I had obtained the source code, I had to go hunting for the assemblies (there were multiple versions involved) in order to get the references corrected so  the project would even compile.  This makes getting set up to modify an existing application a long and arduous task.  Then, too, the controls have to be installed on your build server in exactly the same location as on the development box or your builds will fail, as well.

Instead of doing it that way, try creating a folder, in your project, that contains all the third part assemblies required by the application.  If you like, you can even subdivide them by vendor, purpose, or whatever in sub folders beneath.  Then, set all your references to the assemblies in those folders.  Make sure, too, that these assemblies get checked into source control along with your source code.

Now, when a new developer goes to check out the code to work on it, all the dependencies are there, working, and a clean compile is only a click away.  Same goes for the build server.  You won't have to worry about installing the dependencies there, at all.

Tags:

.Net | Productivity

Hosted Services

by mgordon 23. April 2008 07:25

Some time ago, I was looking for a way to set up a family calendar so we could all keep track of one another's plans and schedules.  Being a typical, time challenged family of the new century we were having a hard time keeping track of all our activities.  My first thought was to try to whip something up from scratch, but being the spare-time-challenged individual that I am made that a difficult proposition.

Around that time, I had listened to Scott Hanselman's HanselMinutes episode about Google Apps and Windows Live Custom Domains.  I immediately went to find out all the details.  The idea with both of these services is that you obtain a domain name and then configure that domain to be redirected to one of these companies' properties for services like email, productivity applications, calendaring, blogging, etc.  You can check out Microsoft's offerings  and Google's .  I elected to sign up for Google Apps.  I found the signup and setup process extremely simple though it does require that you have access to change and knowledge about the DNS entries for your domain.  The instructions on the web site are easily followed, though.  Initially, the URL's to the various features are quite cryptic, but Google provides a way for you to replace these paths with sub-domains, if you choose.  For example, the path to my domain's GMail login was something like https://www.google.com/a/codespot.com/ServiceLogin?service=mail&passive=true&rm=false&continue=http%3A%2F%2Fmail.google.com%2Fa%2Fcodespot.com%2F&ltmpl=default&ltmplcache=2 and by taking advantage of a sub-domain, it became mail.codespot.com.

The services allow you to set up an eco-system for any group of folks that you like.  You create accounts for those you wish to have access to your domain's offerings and the content you keep there can either be exclusive to the members of the group or shared with all netizens. 

All my family members now have a personal calendar and any of us can choose to overlay our calendar with the events schedules on the calendar of another family member.  We can choose to be notified via email or SMS of any event we choose.

With both offerings, the best feature is that it's all for free.

Tags:

General | Productivity

Linq Result to DataSet - Revisited

by mgordon 23. April 2008 06:57

Some time ago, I posted a link to an article explaining how to obtain a DataSet from the result set of a Linq query.  I've continued to look for elegant, natural solutions to this problem and I've found one that appeals to me more, here.This solution, similar to the last, uses extension methods to get the job done, but I like the way this one is implemented better.

Tags:

.Net | Database | Linq | Sql Server 2005

Setting Up a Build Server - Part 4 (StatSvn)

by mgordon 2. April 2008 05:52

So, now we have a build server including source control, a mechanism to monitor our source code and trigger builds and a utility in place to allow us to automate actions like compiling, running unit tests, deploying files.  We can also look at the Cruise Control web application to track our builds, their progress, success, failure.

This time, we'll take a look at a package called StatSvn which tracks the changes made to your code base and generates informative reports from what it sees.    Let's dig right in to setting it up.

First, you'll need to download the software from here.  You'll be downloading a zip file which needs to be expanded into a directory on your build server.  In that directory, you'll find a jar file, so you know right away you need to install the Java runtime on the build server.  You can find the runtime here.  After insuring the runtime is installed, you'll need to create a new directory on the server to contain a running current version of your code base.  StatSvn will be using the code in this directory to generate its reports from.  After creating the directory, check the current version of the code base into it.

We'll be calling StatSvn from our build script.  Since there are several steps to working with StatSvn, I create a bat file containing all the steps and then call it from the build script.  So, create a new bat file (Stats.bat) on the root of a drive on the build server.  It should contain something like the following.

c:
cd c:\stats
del *.html
del *.png
cd "c:\StatCode"
svn up
svn log --xml -v > svn.log
cd c:\stats\
java -jar statsvn.jar "c:\StatCode\svn.log" "c:\StatCode"

Here, I'm changing the current drive to c: and then changing the current path to where I unzipped the StatSvn jar file.  When the jar is executed, it creates several html and ping files in its current directory.  I start, then, by deleting all these from the last run.  Next, I change the current path to point to the directory into which I checked out the current version of my code base.  I then call the Subversion command line utility, svn, and pass it a parameter or "up".  This tells subversion to update the local copy of my code with what's on the server.  Now, I call the Subversion command line, again, and ask it to create a log file for me, called svn.log, in XML format.  This file will be created in the same folder my source code is in.  Last, I change the local path back to where I unzipped the StatSvn archive and execute the jar file telling it where my log file and code base is.  The result of this is the creation of html and image files in the StatSvn folder containing reports about my source code.  Let's tie this process into the build script.

Last time, we looked at using the <exec> element in our NAnt script to execute command line statements.  To add the batch file to our build, therefore, all we need to do to add a call to our batch file is to either create a new target or use an existing target and add this task inside it.

<exec program="C:\stats.bat"></exec>

Now, how do we view the generated reports?  Generally, you'll want to make sure IIS is installed on your build server and create a virtual directory that points to the StatSvn folder.  Be sure to set the default page to index.html.  Once you do this, you'll be able to navigate all the generated content with your browser.  There are examples of the generated reports on the StatSvn web site for you to check out.

Tags:

.Net | Productivity

Setting Up a Build Server - Part 3 (NAnt)

by mgordon 1. April 2008 13:21

So far, in this series,  we've installed and configured Subversion for source control and installed and partially configured CruiseControl.Net as a continuous integration server.  This time, we'll look at a utility called NAnt which provides a declarative way of specifying tasks that are to be performed - specifically, tasks to be completed during a build.

A NAnt script is an XML document containing nodes that the NAnt engine understands.  By carefully creating a NAnt script, we can have the tool automate many tasks for us at build time.  NAnt can be downloaded from the site,  here and the documentation is here.  NAnt is powerful, out of the box, but its functionality can be extended by also obtaining the NAnt Contrib library which contains additional tags you can use in your script file.

To install NAnt, Download the zip file containing the binary files and unzip the archive to a folder on your hard drive.  You'll probably want to make this folder as shallow as possible (c:\nant) since you'll be specifying this path in CruiseControl.Net setup and possibly on the command line for testing you scripts.  As an alternative, you can place the code anywhere and add the path to the bin folder beneath the root folder to the path variable. 

Next, download the NAntContrib library and unzip the contents of the downloaded archive to a different folder.  Now, copy the contents of the bin folder beneath where you unzipped NAntContrib into the bin folder beneath where you placed NAnt.  You should now be ready to start building scripts.

NAnt looks for its instructions in a file with a .build extension.  As I said before, this file contains XML.  I won't try to teach you all there is to know about NAnt (I don't know it all, anyway), but I'll try to give you enough of the basics to get you stared on a firm foundation.  The root node of a build file looks like this.

<?xml version="1.0" encoding="utf-8" ?>
<project name="Rtc.Fx" default="CopyFiles" basedir="." xmlns="http://nant.sf.net/release/0.85/nant.xsd">
</project>

The root node is a <project> node and specifies the project name and a default task.  The build file is broken up into separate tasks and the default attribute indicates which task to begin with.  There may be certain values we'll want to use over and over in our build file or that we'll want to calculate.  We can work with these as properties.

  <property name="month" value="${datetime::get-month(datetime::now())}"/>
  <property name="day" value="${datetime::get-day(datetime::now())}"/>
  <property name="year" value="${datetime::get-year(datetime::now())}"/>
  <property name="hour" value="${datetime::get-hour(datetime::now())}"/>
  <property name="minute" value="${datetime::get-minute(datetime::now())}"/>
  <property name="second" value="${datetime::get-second(datetime::now())}"/>
  <property name="buildDirName" value="build_${month}-${day}-${year}_${hour}-${minute}-${second}"/>
  <property name="sourceDirName" value="source_${month}-${day}-${year}_${hour}-${minute}-${second}"/>

Insert these inside the root node.  As you can see, we're calling available functions to obtain values for parts of the current date and time.  These are stored in parameters have the name specified in the name attribute.  We've calculated names, here, for our source and destination folders.  Often, you'll want to save versions of code on your build server for a while and creating new folders with the date and time in the name is an easy way to identify when the snapshot was pulled down for a build.  Now let's actually do something.  Let's start by pulling the current version of our source code down for a compile.

<?xml version="1.0" encoding="utf-8" ?>
<project name="Rtc.Fx" default="CreateTags" basedir="." xmlns="
http://nant.sf.net/release/0.85/nant.xsd">
  <property name="month" value="${datetime::get-month(datetime::now())}"/>
  <property name="day" value="${datetime::get-day(datetime::now())}"/>
  <property name="year" value="${datetime::get-year(datetime::now())}"/>
  <property name="hour" value="${datetime::get-hour(datetime::now())}"/>
  <property name="minute" value="${datetime::get-minute(datetime::now())}"/>
  <property name="second" value="${datetime::get-second(datetime::now())}"/>
  <property name="buildDirName" value="build_${month}-${day}-${year}_${hour}-${minute}-${second}"/>
  <property name="sourceDirName" value="source_${month}-${day}-${year}_${hour}-${minute}-${second}"/>

  <target name="GetSolution"  description="Get Solution Files from Subversion">
    <mkdir dir="c:\build\source\Rtc.Crm\${sourceDirName}"/>
    <svn-checkout
      destination="C:\Build\source\Rtc.Crm\${sourceDirName}"
      uri="svn://localhost/rtc_repository/RTC/Rtc.Crm"
      quiet="true"
      username="uname"
      password="pword"
      />
  </target>
</project>

Tasks, in NAnt, are called targets.  More on why they maybe called this and how to control the flow of execution in a minute, but for now look at the target we added.  It has a name of GetSolution and an optional description.  Inside each target, we can specify zero or more actual action that need to take place.  In this example, a new directory is being created to place the source code into by using the mkdir tag.  It will have the name we calculated in the parameter, above.  In NAnt, a property(Variable, Function all, anything that returns a value) is accessed by surrounding it with ${}.  Once the directory is prepared for our source code, we'll ask Subversion to check our code out into the new folder by using the svn-checkout tag.  We specify a destination for the source code, the uri path inside the repository where the source code is located, a user name and a password for an account having access to the repository.  When this tag is encountered, the code will be checked out for us.  Now, let's specify another target that will build our code.

<?xml version="1.0" encoding="utf-8" ?>
<project name="Rtc.Fx" default="CreateTags" basedir="." xmlns="
http://nant.sf.net/release/0.85/nant.xsd">
  <property name="month" value="${datetime::get-month(datetime::now())}"/>
  <property name="day" value="${datetime::get-day(datetime::now())}"/>
  <property name="year" value="${datetime::get-year(datetime::now())}"/>
  <property name="hour" value="${datetime::get-hour(datetime::now())}"/>
  <property name="minute" value="${datetime::get-minute(datetime::now())}"/>
  <property name="second" value="${datetime::get-second(datetime::now())}"/>
  <property name="buildDirName" value="build_${month}-${day}-${year}_${hour}-${minute}-${second}"/>
  <property name="sourceDirName" value="source_${month}-${day}-${year}_${hour}-${minute}-${second}"/>

  <target name="GetSolution"  description="Get Solution Files from Subversion">
    <mkdir dir="c:\build\source\Rtc.Crm\${sourceDirName}"/>
    <svn-checkout
      destination="C:\Build\source\Rtc.Crm\${sourceDirName}"
      uri="svn://localhost/rtc_repository/RTC/Rtc.Crm"
      quiet="true"
      username="build"
      password="build"
      />
  </target>

  <target name="build" description="Build Core Solution" depends="GetSolution">
        <exec program="c:\program files\Microsoft Visual Studio 8\Common7\IDE\devenv.com" commandline="c:\build\source\Rtc.Crm\${sourceDirName}\Rtc.Crm.Sln /build Release"></exec>
  </target>
</project>

We've added a new target, now, called build.  Notice the depends attribute.  This attribute, in effect, says don't run this target until the GetSolution target has finished.  This is how you can control the flow of your targets.  If we make the build target the default (on the root node), NAnt will come to the build task and see that it can't execute it until GetSolution has run, so it puts build on hold and goes to execute GetSolution.  Once it's finished, the build target is executed.  In the build target, we're using the exec tag to build our code.  You could use compile tags in the NAnt libraries to compile your code, as an alternative (or MSBuild, the C# compiler or whatever), but I have found that it's hard, sometimes, to keep the .Net version and the NAnt version in sync.  For this reason...stability...I prefer to use the exec task and specify a command line to execute.

From here, you can use any of the provided tags to round out all the tasks you want to automate in your build.  You could copy the binaries just compiled to a deployment location, run NUnit tests, create a cd image file or whatever is needed.

Integrating NAnt into CruiseControl.Net

Last time, we left our CruiseControl config file looking like this...

<cruisecontrol>
<project name="MyProject" >
      <webURL>http://servername/ccnet/</webURL>
      <triggers>
        <intervalTrigger seconds="60" />
      </triggers>
      <modificationDelaySeconds>60</modificationDelaySeconds>
      <sourcecontrol type="svn">
        <executable>c:\program files\subversion\bin\svn.exe</executable>
        <workingDirectory>c:\build\source\ProjectName</workingDirectory>
        <trunkUrl>svn://localhost/repository/path/to/project</trunkUrl>
        <username>uname</username>
        <password>password</password>
      </sourcecontrol>
      <tasks>
        <nant>
          <executable>c:\nant\bin\nant.exe</executable>
          <baseDirectory>c:\build\source\ProjectName</baseDirectory>
          <buildFile>c:\build files\Project.build</buildFile>
          <targetList>
            <target>TargetName</target>
          </targetList>
          <buildTimeoutSeconds>600</buildTimeoutSeconds>
        </nant>

      </tasks>
      <publishers>
        <xmllogger />
        <statistics/>
      </publishers>
    </project>
</cruiseControl>

We can, now, complete the section in red.  The executable tag contains the path to the NAnt executable and the buildFile tag the path to the build file we just created.  The baseDirectory is where NAnt will do its work and the targetName is the name of the target within the build file to execute first.  You can also specify a timeout for the build.

So, the config file as it stand now will check Subversion every 60 seconds to see if there have been any changes checked into the repository.  If so, CCNet will pause for 60 seconds to give the developer enough time to complete checking in all their code and then initiate a build.  Now CruiseControl will look at all the tasks in the tasks section of the config file.  Here, there is one and that is to execute our NAnt script.

If you code the defaults when you installed CC.Net, you should be able to browse to http://buildServerName/ccnet and take a peak at the CCNet portal. Make sure the Cruise Control service is started!  The portal looks like this.

ccnet

From here, you can look at the progress for all your projects and builds.  Notice the link in the upper right to the documentation.  There is also a link on the far left to download CCTray, which is a utility that runs in the system tray and lets you know, at a glance, the health of your builds and double-clicking on the icon will open a utility that offers more functionality.

Next time, we'll look at how to get metrics from your subversion database as part of your build process.

Tags:

.Net | Productivity

Powered by BlogEngine.NET 1.5.0.7
Theme by Extensive SEO