Enterprise Etc Adventures and Insights of a .Net Developer

I’ve had my Windows Phone 7 for about a year, now, and I’ve found it to be the useful, flexible device I’ve owned.  Plus, it’s fun to use.  Over the course of the past year, I’ve laid several other devices and programs aside and have, instead, allowed my phone to take over their former roles.  Below, I’d like to talk a bit about the various ways that my Windows Phone has become the central device in my life.

Social Integration

I find that the integration of Facebook, LinkedIn and twitter into my phone has virtually eliminated the need for me to ever have to open any of these sites in a browser.  Granted I’m not a super user of any of these sites, but other than profile updates and the like, the phone offers all I need.  I’m notified of any Tweets directed at me or any mentions of me on Facebook, any person from my contact list that I pin to the start screen allows me to immediately see a change in their status and the People Hub allows me to browse a consolidated feed of all the news from all the sites.  In addition, I can upload photos to my accounts, post a status or check in.

Google Analytics

It’s fun to follow the traffic and popular content on my blog.  Used to be, I had to browse to the Google Analytics site to get this information.  However, I know open an app on my phone, Phonealytics Free, and see a graph of my hits by date as well as by post.  A nice application that allows me to see what I want to with a couple of taps and swipes.

GPS

One device that I used to keep in my Jeep was a Gamin Nuvi GPS.  The maps were getting old and the windshield mount had just broken.  About this same time, I became aware of the Garmin StreetPilot app for windows phone.  I downloaded the trial and gave it a try.  I was amazed at how much better the features were in the phone app compared to the device.  Within a week of obtaining the trial version of the application, I upgraded to the full version and have used it daily since.  The app offers turn-by-turn directions via voice (with street names), traffic information and delays, photos of highway exits and information on what lanes you need to be in for the next turn.  You can save favorite locations, as well.   There is never an issue with the age of the maps or information about places of interest as the maps are downloaded as needed and the app uses Google Search to find locations.

Music and Podcasts

A second device I always kept with me was a 128 GB Zune.  I kept my entire music collection on the device and periodically updated podcast feeds to it.  My Jeep has a plug for auxiliary inputs and I’d just plug the Zune into the Stereo and listen to this media as I drove.  The Windows Phone Marketplace now has a podcast section that allows me to subscribe to podcasts on the phone.  I can specify how many episodes to keep on the phone and the phone automatically updates my subscriptions over wifi when it has access to it.  This has allowed me to always have fresh podcast content without having to worry about plugging a device into my PC for syncing.

It would be impossible for me to keep my entire music collection on my phone, but it would be great to have access to all of it via streaming.  I recently found the tversity application in the marketplace and gave it a spin.  The app requires that you install an application on a PC containing your content and that application exposes the titles to the phone app.  I already had my collection on a Homeserver, so I installed the tversity application and configured it to expose not only my music collection, but also my videos.  Now, I can stream my music to my phone wherever I am, over 3G.  The video is a bit jerky over 3G, but works great over WiFi.  Another device I no longer need.

There are many other apps that I used almost every day that make my life simpler.  Access to cloud storage from any device with the SkyDrive app.  Access to OneNote notes from anywhere make it easy to stay organized.  The Lync app allows me to stay in touch with coworkers via text and voice.

My phone is easily one of the best purchases I’ve ever made and I’m excited to see what Microsoft does with the platform going forward.

For the past several months, we've been using a build server on our project for continuous integration.  I've been involved with teams, in the past, that used build servers, but this particular setup has proved to to be a real asset and the experience of using it day-to-day has been a pleasure. 

I'd like to share how we built the server.  I'll make several posts over the next few days to address the various parts that work together.  Along the way, I'll discuss version control (subversion in our case), Cruise Control .Net, NAnt and StatSvn.

What We're Trying to Accomplish

Why would you want a build server, anyway?  The primary benefits of a build server are continuous integration and automation.  Let's say you have a team of five and all are working on the same project.  Each has a piece of the application that they're working on.  As they develop, they test the code they're responsible for.  Who, though, is responsible for making sure that all the pieces work together?  The reality is that at any given time each developer's machine has a unique version of the application that's different everyone else's.  The purpose of the build server is to periodically take the code that exists in the source repository and process is to verify the code all works together.  In other words, the health of the code (how well all the parts integrate) is always known.

There are many functions the build server can perform.  It can do a compile, execute test suites, generate an installer or even a CD image.  Crucial to the whole process is source control, so let's look at getting started with Subversion.

Subversion

All source code repositories have their own take on how best to solve the problem of keeping up with all the changes made to to the code base.  Some require the developer to lock a file while it's being worked on, while others allow changes to be made by anyone at any time.  Subversion falls into the latter category.  Basically, a developer grabs a snapshot of the current version in the repository, makes his changes and then pushes them back up.  Committing of changes happens in a transaction so the commits are always stable.  Subversion takes the version being committed and merges it into the current version in the repository.  This means that manual merging is greatly reduced.  On the occasions when Subversion can't figure out how to merge the changes, the developer is notified and is give an opportunity to manually do the merge. 

Installation

You can obtain the subversion installer here.  I downloaded the installer, svn-1.4.6-setup.exe, and executed it taking all the defaults.  At this point, all the binaries are installed on your system, but Subversion hasn't been configured to run as a Windows Service.  To set it up to run this way, execute the following from a command line.

C:\>sc create svn binPath= "\"c:\program files\subversion\bin\svnserve.exe\" --service -r c:\repos" DisplayName= "Subversion Server" depend= Tcpip

This command will create a Windows service named "Subversion Server" that runs svnserve.exe when it's started. 

Subversion Client

Regarding a client that you'll use for interacting with your Subversion server, you'll need to decide whether you prefer to have the client integrated into Visual Studio or not.  If having everything available within the IDE is you bent, then you'll likely prefer something like AnkhSVN.  Personally, I'd rather do my source repository work outside the IDE and prefer to use TortoiseSVN which installs as a shell extension.  Either way, you'll need to create a repository and then connect to it with your client.

Creating and Configuring a Repository

On the machine on which the Subversion service is running, create a folder in which you'll be storing your code repository.  Then, go to a command line and execute the following.

svnadmin create c:\path-to-repos

This will create a repository for you in the directory you created.  Now, we need to configure access to the repository.  Go to the folder you created, before, and you'll find that three files have been created there.  First, open the svnserve file.  All the lines in the file are currently commented out.  You'll need to modify at lease three of these lines.  Find the line that contains "anon-access".  This line controls what access users connecting to the repository anonymously have.  Your options are "read", "write" and "none".  In my case, I set this value to "none".  Next, find the line that contains "auth-access" and set its value to either "read", "write" or "none".  this line controls what access authenticated users will have.  For the last change, find the line that contains "password-db" and remove the "#" character and the space that follows it to un-comment the line.  This tell  Subversion to look in the passwd file to find a list if valid users of the repository.  Save your changes to this file.

Now, open the passwd file.  All lines in this file are commented out, as well.  Beneath the line "[users]", you will need to create a list of valid users for the repository in the form "username = password".  Save your changes.

Accessing the Repository From a Client

Whichever client you choose, you'll need to specify a URL and your credentials.  URL's for subversion are of the form svn://server-name.  For credentials, you'll need to use one of the user-name / password combinations you created in the passwd file, earlier.  At this point, you should be able to check out (get a copy of the current version) and commit (check in) to and from your repository.  If, however, you're having problems reaching the server you'll want to check that the Subversion service is running and you also may need to configure svnserve.exe as an exception in windows firewall.

That's it for Subversion.  Next time, we'll set up CruiseControl.Net.

Imagine being given a large project to complete and several months in which to do it.  You've been given a vision of the finished project and little else.  How do you make the best use of the time?  How to you keep every member of your team engaged for the full duration?  How do you measure your progress as the time slips past you?

Consider the ways in which iterative development can come to your rescue in this situation.

Productivity - When you want to write a system that needs great throughput, often you'll feed that system with some type of queue.  As a task is finished, the next is pulled from the queue and work continues.  No negotiation is requires, no prompting or polling.  The next work item is is there and just waiting to be done.  Many of the developers I've spoken with say they like working in a  similar manner.  Queue up the work so it's waiting for me to do it, and I can blow through the tasks at a much faster pace. 

An iterative development process works in much the same way.  While work is being done on the current iteration, any additional requirements ro issues are being queued up on a task list, prioritized and scheduled.  An important point, here, is that it isn't the developer who's having to deal with  finding the next task, negotiating with management about the length of time the task will require to complete or to hash out the specific requirements.  When the current iteration finishes, the tasks for the next are ready to be undertaken by the developer.  This means that several of the things that can steal time from those writing the code are eliminated.  There will always be the need to converse with users about specifics or to aid in providing estimates on the amount of effort required for a task, however.

Organization - I've worked in environments where requirements or issues could be brought in at any time and justified an immediate change in priorities and focus.  At the end of the day, much of your productivity goes down the drain as you take the time to switch contexts - unwind the task you're working on and dig into the one being presented.  With an iterative process in place, most of these issues can be placed on the queue and scheduled for a later iteration.  This forces an evaluation of how immediate a need really is and cuts down on context switches required which keeps the developer focused and productive.

Morale - No one like to feel they're spinning their wheels - making huge efforts with little progress to show for it.  The simple act of setting an obtainable, short-term goal both helps all involved to feel that progress is begin made and also allows for the easy gathering of metrics that establish what point in the scope of the entire project has been attained. 

Flexible Scheduling - There are almost always external factors such as outside vendors and the availability of resources that affect when certain parts of a project can be addressed.  With an iterative approach, the impact of these factors can be mitigated, to a degree, by putting off these parts of the application until they can be addressed.

When .Net was introduced, one of the most powerful language features has been delegates which provide a way to pass around, not only data values, but also chunks of implementation and logic.  Consider how often you need to add a button to a web form or windows form and have some code execute when the button is clicked.  In Visual Studio, you can simply double-click on the button in the designer and the IDE will generate a new method for you, and wire that method to the event.  The generated code contsructs a delegate and passes it to the button so it can be called by the button at the proper time.  In effect, it tells the button, "When you're clicked on, execute this chunk of code".

Delegates have come a long way since the 1.0 days.  Consider the following code for a Windows Form class.  In this example, we'll look at the evolution of the delegate.

  public partial class Form1 : Form    
  {        
public Form1()        
{            
InitializeComponent();             

         //DotNet 1.0 and 1.1 - had to specify the implementation in a separate method            
this.button1.Click += new System.EventHandler(this.button1_Click);                        

        //DotNet 2.0 - Anonymous Methods.  Can declare implementation inline.            
this.button2.Click += delegate            
{
                MessageBox.Show("Clicked Button 2");
        };             

        //DotNet 3.5 - Declaring implementation inline using Lambda Expressions
        this.button3.Click += (sender, e) => MessageBox.Show("Clicked Button 3");

    }

    private void button1_Click(object sender, EventArgs e)
    {
            MessageBox.Show("Clicked Button 1");
    }
} 

For Button2, I've created the delegate using the anonymous method syntax introduced in .Net 2.0.  With it, you don't need to declare a separate method containing the implementation.  Instead, where the delegate is declared, you can also specify the implementation.  This makes the code both more brief and also keeps the delegate declaration and its implementation located together, making the code more readable.

In the latest .Net release, Lambda Expressions were introduced.  While the name may sound scary and complicated, they're really nothing more than a third syntax for specifying delegate implementations.  As you can see in the implementation for Button3, I've specified the familiar parameter list of (sender, e).  Also, I didn't need to specify the types of the parameters though you could if you wanted to.  The compiler will figure out the types for us based on the context of the expression. As Scott Guthrie explains in this post, the parameters' types are also known by the intellisense engine so we can get intellisense on the parameters. The parameter list is then followed by a "=>" symbol and finally the implementation itself.  Once again, more concise and far more readable than the previously introduced syntax.

I can barely imagine what LINQ queries would look like had the latter two syntaxes not been introduced for delegates (especially the Lambda Expression syntax), but this doesn't mean the improvements are only useable in Linq queries.  As I've demonstrated, they are useful and even preferable any time a delegate is required. 

I'm currently working on a project that began in March of last year.  A friend of mine is managing the project and after he brought me on, we decided to try and make things as pleasant and as fun as possible for the project team.  I purchased a couple of novelties that we awarded developers when they broke the build or were deemed the most productive.  We blazed through requirements and watched features get implemented swiftly.  I was having a ball and it seemed the whole team was enjoying working on the project. 

Around mid summer, though, something happened.  Several, previously unknown, requirements were uncovered and while they were added to the project, the timeline was not adjusted.  I marveled that in the space of about a week both motivation and productivity dropped noticeably.  I felt my own attitude about the project begin to change and in asking myself why I identified a few things that motivated me and a few that did not.

What Motivates Me

• An opportunity to learn new technologies is always a motivator.  Looking back, I've always been willing to put forth the extra effort and spend the extra time to stay as productive as possible while learning about new tools, languages or language features.
• Trust.  When a task has been delegated to me and trust has been placed in me to get the task done, I've always risen tot he challenge.  In fact, any gesture that communicates, "I believe in you and your abilities." is bound to motivate me.
• Organization.  When a project is well managed and organized so that I can concentrate on what I do best (design and build software), I find I have more energy and apply myself better to the tasks at hand.  

What Robs Me Of Motivation 

• Repetitive tasks.  Copy / paste, boiler plate code...yuck.  Can't we automate this task and move on to something else??
• Micro-managing.  I think of myself as a professional and I don't think its too much to ask to be treated that way.  Ask me if I can perform a task.  If I tell you I can perform it, let go of the task and give it to me.  I'll give it back when it's completed.  I promise.
• Surprises.  Whether it's formal or not, there is always a plan.  When I start a project, I may not know precisely what the finished product will look like and I may no be able to tell you precisely when it'll be finished but you can bet I have a plan.  I need to know what constitutes a finished product and have some idea of how long the task will take.  When the environment in which I'm working or the projects requirements are in a constant state of flux, it drains me of energy and motivation.  Will the section of code I'm working on, right now, just be thrown away?

I'm sure you have your own similar experiences.  What has motivated you, in the past?

I've been exploring some of the new language features that will be available in the coming version of .Net.  I continue to see things that look like they have been borrowed from dynamic languages.  Another to add to the list is automatic properties.  In C#, you'll be able to define properties like this.

public string FirstName { get; set;}

This is functionally equivalent to the following:

private string _FirstName;
public string FirstName
{
    get
    {
        return _FirstName;
    }
    set
    {
        _FirstName = value;
    }
}

This will be a real timesaver for those not using a tool such as CodeRush where creating he above block of code is reduced to three keystrokes (ps<space>) and then typing in the name of the property. 

Now, look at how the equivalent is accomplished in Ruby.

attr_accessor: FirstName

Instead of specifying get; and set;, you specify attr_writer for a write-only property and attr_reader for read-only.

With the release of the next version of Visual Studio and the .Net languages, a new feature called Extension Methods will be available.  In a general sense, these allow you to add functionality to classes without having to modify their source including any class in the .Net Framework.  This is a very similar idea to what is offered in a dynamic language such as Ruby, but in the strongly typed .Net languages.

 For example, Ruby has a string type.  If, for some reason, we wanted to add a method to the string class that would return the string wrapped in html that would render the string in a particular color, we could do something like this.

This would return the string <font color='#AAAAAA>fido</font>

In C#, we can accomplish a similar thing this way.

Here, I've added a to_i() method to the .Net string class that converts the string value to an integer if it can and throws an exception if it can't.  What triggers the desired behavior is the inclusion of the this keyword in the method signature.  This syntax tells the compiler that the method is to be added to the string type since it follows the this keyword.  The new method can be used on the string class anytime my extension class is in scope.  For example, I could have added my Extensions class in a different namespace and anytime that namespace was included with a using or imports statment, my extension method is available.

Certainly, this is a more brief syntax than int.Parse("423") and feels more natural.  I can recall several situations where I've had to string together commands from various classes in a single statement.  Extension methods could help out with creating convenience methods to replace complicated syntax in these situations.

Scott Gutherie discusses how extension methods were used to add functionality to classes for use with Linq, here.  When the Linq namespaces are included, standard classes are augmented with new functionality that make them easier to use in conjunction with Linq.  In thinking about how I might use these, one idea that came to mind was in a situation where I had a data transport class.  It needs only to be able to contain the data when it's moving between layers, but when it arrives in the business layer from the data layer, it would be nice to be able to add functionality to the class.  This would keep the class nice and tight as it is passed around, but when you need to perform some action on the data contained within, extension methods could be used to add that functionality.  In a way, this technology allows you to have a class function differently based on the "mode" or context it happens to be in at any given time.  Very snazzy!

In my current contract, it's the typical story with compressed timelines and pulling off miracles, so I haven't had the opportunity to make use of Ruby for a "real" production application.  However, we're in the midst of doing some work in migrating data from a legacy system into a completely new database schema.  All imagined challenges apply, I assure you.  Part of this exercise is repeatedly bouncing the data against the new schema, applying validations to the data to make sure all the new referential integrity holds true in the new system and reporting on data that isn't fitting into the new system properly.  I took this as an opportunity to apply some Ruby scripting as a means of automating this part of the project.

 Having played with the DBI library a bit before starting, I knew that I wanted to abstract some of the interaction with it away so I wrapped it like so.

  ########## Class to abstract the server connection ##########
  class SqlServer
   
    def initialize(connectionString)
        @dbh=DBI.connect("DBI:ADO:Provider=SQLOLEDB;" + connectionString)
        @dbh["AutoCommit"] = true
    end
   
    def execute(sql)
        statement = @dbh.prepare(sql)
        statement.execute
    end
   
    def fetch(sql)
        statement = @dbh.execute(sql)
        statement
    end
   
    def disconnect
        @dbh.disconnect
    end
  end

I needed to be able to pull fresh data from other databases into my conversion database, so I abstracted that task, as well.

In my case, all the columns were varchar, so I didn't have to fuss with the format of the data in the insert statements.  With DBI, however, it's possible to get the meta data about each column to decide whether the data needs to be quoted or not.  Now, I needed to be able to pull data about how the load went and any exceptions that were encountered.  That data needed to be saved into CSV files so they could be attached to emails.  The next class was written to do this.

 ########## Class to pull statistic about the load into csv files ##########
 class StatFile
 
    def initialize(connString)
        @server = SqlServer.new(connString)
    end
 
    def GetStats(filePath, sql)
        statement = @server.fetch(sql)
       
        output = File.open(filePath, "w")
       
        statement.fetch do |row|
            output.puts row.to_a.join(",")
        end
       
        output.close
    end
 end

 With the above classes, I was able to script the tasks at hand something like this.

Pull.new("tableName").CopyTable #Copies the table from one db to another

 conversion = SqlServer.new("Data Source=server;initial catalog=database; trusted_connection=yes")
 conversion.do("EXEC ConversionSproc")

 stat.Getstats("c:\\Stat_Files\\exceptions.csv", "select foo1, foo2 from exception_table where error='bad' group by description")

As I've stated before, I'm not planning to use Ruby to put food on the table at this point, but I would like to get some experience using the language if only to get the chance to think about things differently.

 UPDATE:
After using the above scripts a bit, I found that when executing some rather long running stored procedures, I was receiving a timeout.  It was then that I realized there was not way to set the CommandTimout property on the database connection through DBI.  After hours of searching, I was able to gleen enough information to overcome the problem.  I added the following line to the bottom of the initialize method of the SqlServer class:

  @dbh.handle.instance_variable_get(:@handle).setproperty('CommandTimeout', 7200)

Of course, you can specify whatever value you like for the timeout or even parametize the value.

For the past few weeks I've been kicking the tires on the Ruby language and the Rails framework.  This post is not meant to be a review of either, but I thought I'd note some of my impressions for other .Net developers that have been hearing the buzz and have not had the time to take a look.

First of all, I'm not planning to make a jump from ASP.Net to rails anytime soon.  However, I think there is always benefit in exploring other technologies, approaches and points of view and my look at these things was primarily motivated by curiosity.

The Ruby Language
I love this language.  Scott Hanselman has commented on how easily intent can be expressed in Ruby and Wilco Bauwer likes its rhythm.  Writing logic in it feels really natural, to me, and it seems that whenever I need a particular piece of functionality, it's available and easy to invoke without having to jump through hoops.  Without casting a vote for either dynamic languages or statically typed ones as the be-all-and-end-all, I have to say that if I were to be tasked with creating a domain specific language I'd certainly go for Ruby as my language of choice to implement it in.  It lends itself well to that type of exercise.  After all, to an extent Rails is a DSL for the web application space.  Some day to day tasks would lend themselves well to being implemented in Ruby.  Command line tasks, in particular, could be easily handled in Ruby.  The language (as implemented for Windows) supports calling into the win32 API and COM.  It does not contain any support for building windows UI, though you can do that by calling the windows API (if you're nuts) or by making use of one of the many Ruby .Net bridges that are freely available.  Just don't expect the performance of the bridge to blow you away.

Another thing that impressed me was the number of dynamic language features that are making their way into .Net languages.  Anonymous methods, continuations and extension methods all seem to have their roots in dynamic languages such as Ruby. 

Ruby is easy to learn and the code is easy to read.

Rails Framework
As one who appreciates the value of convention and likes having a process in place to grease the wheels of progress, I find Rails to be a very comfortable approach to use for web development.  As stated on many Rails-Centric sites, Rails is about "Convention over configuration"  and "DRY" or "Don't Repeat Yourself" meaning don't duplicate code.  Typical frameworks offer abstractions to make the most commonly required functionality easier and faster to use.  Rails does this well, but adds code generation and a type of adaptability that could only be achieved by using dynamic language such as Ruby.  There is also a substantial number of tools to aid the developer with ancillary tasks such as deployment and testing.  In general, Rails forces the developer to use the Model-View-Controller pattern to build their web application.  Few would argue with this approach.  ActiveRecord, which is an ORM library, is used for database access.  ActiveRecord is the stated inspiration for the Subsonic library written in .Net and greatly simplifies database access.

Many proponents of Rails claim extraordinary gains in productivity can be achieved.  I can see how productivity can be increased greatly for an experienced Rails developer, but be warned that there is a lot to remember and new Rails developers may find they are spending a great amount of time in the Rails documentation. 

There are a few IDE's out there that support Rails development.  I preferred RadRails and have been using it for the past few weeks with good success.  I found that Rails, as many have claimed, brought some fun back to the development of web pages. 

The one area I found to be a significant pain was in the deployment story.  Rails is not thread safe.  This means that each web request cannot be handled by only a new thread, but a new process.  I searched, for some time, and found some solutions using FastCgi which basically spins up several instances of a web server and feeds requests to them.  There seem to be some better solutions on the horizon , but I was not able to find what I would consider to be a "clean" solution to use immediately.

I've gained a number of insights, so far, from this exercise.  I don't see myself building my next business application in Rails (but, who knows) or even being gainfully employed as a Rails developer, but the lessons learned will continue to open my mind and make me a better developer.

I'm in a situation, right now, where scope creep is a problem.  It's frustrating, without a doubt, but I started wondering why it was such a source of stress and made some observations.

I'm a person who likes things very ordered and I suspect that most good developers are successful at what they do because they, too, like things to be organized and ordered.  If your going to be using a waterfall type approach to manage the project, ordered means having to know, up front, what all tasks need to be completed and how long each will take.  Without this information, there's no way to know if the project is on track or not as it progresses.

Business people don't seem to work well, that way.  Typically, they'll get their hands on a nearly finished product and start finding exceptions to the rules that were first specified when the design was being made.  Bottom line, here, is that there will almost always be late-breaking requirements and modifications that will be specified.  Knowing this can get you half way to a point of being stress free.

The other half of the frustration stems from the fact that these late-breaking revelations are contrary to the method used to manage the project.  The whole process used to manage the project gets stood on its ear and the project goes into a kind of suspect status because our process is not equipped to handle this disruption of the project flow.  This causes stress on the development team and causes the project stake holders to get nervous.

If we accept as fact that requirements are going to change and that users are going to think of things they didn't realize during the requirements gathering stage of the project, and make that fact a part of the process we use to manage the project, these changes don't come as a surprise to anyone involved and the stress level drops considerably.  To the developer, the change is just another item on the back log of tasks and to the business side of the house it just means they're going to get what they REALLY need.

The key, here, is that you have to start the project out with an agile approach so that everyone's expectations are set from the get-go.  Trying to switch an existing project to an agile method somewhere in the middle is a sure recipe for crash-and-burn.