Wednesday, June 25, 2008

Adding Subversion Ignores from the command line

I use Subversion at work and when I'm managing files from the command prompt, I generally don't enjoy having to sift through a long list of file names with question marks next to them, wondering whether these files should be checked into source control. Folders like "bin" and "obj" and user-preference files have no place in source control -- they just seem to get in the way.

If you're using TortoiseSVN, you can hide theses folder from source control simply by pulling up the context-menu for the un-versioned folder, select TortoiseSVN and "Add to ignore list". However, if you're using the command prompt, it requires a bit more effort. (Over the last few years, I've grown a well established distrust for TortiseSVN, as they shell-overlays can cripple even the fastest machines. I really wish the TortiseSVN guys would release their merge tool as a separate download, if you know a good diff tool, let me know.)

Because the svn:ignore property is stored as a new line delimited list, you need to pipe the results into a text file and edit them in the tool of your choice. When updating the property, I use the -F argument to specify a file instead of supplying the value in the command line.

  1. Get a list of the current ignores and pipe it into a text file:
    svn propget svn:ignore . > .ignore
  2. Edit the list in your editor:
    notepad .ignore
  3. Put the property back in:
    svn propset svn:ignore -F.ignore .
  4. Verify that your ignores work:
    svn st
  5. Commit your changes into the repository:
    svn ci . -m"Updating ignores"

Monday, June 23, 2008

Working with Legacy .NET Projects

My current project at work is a legacy application, written using .NET 1.1. The application is at least 5 years old and has had a wide range of developers. It's complex, has many third-party elements and constraints and lots of lots of code. Like all legacy applications, they set out with best of intentions but ended up somewhere else when new requirements started to deviate from the original design. It's safe to say that it's got challenges, it works despite its bugs and all hope is not yet lost.

Oh, and no unit Tests. Which in my world, is a pretty big thing. Hope you like Spaghetti!!

Fortunately, the client has agreed to a .NET 2.0 migration, which is a great starting place. All in all, I see this as a great refactoring exercise to slowly introduce tests and proper design. Along the way, we'll be fixing bugs, improving performance and reducing friction to change. I'll be writing some posts over the next while that talk about the strategies were using to change our landscape. Maybe, you'll find them useful for your project.

Related Posts:

submit to reddit

Thursday, June 19, 2008

Debugging WebTrends Page Attributes

A few weeks back, I provided a specially constructed link that would allow you to debug HitBox page attributes. I had the pleasure (sarcasm intended) of attending WebTrends training this week, which revealed a similar gem...

javascript:alert(gImages[0].src)

To use, drag this link to your browser toolbar: Show WebTrends.

When clicked, the resulting alert shows all the attributes that are sent to WebTrends SmartSource data collector.

If you want to try it out, Motorcylce USA uses WebTrends.

Update 6-20-08: If you're using FireBug in FireFox, the network performance tab makes it really easy to view the querystring parameters associated with the WebTrends tracking image.

  1. Navigate to your page.
  2. Open FireBug.
  3. Select the Net tab.
  4. Click on the Images button in the menu.
  5. Find the instance of dcs.gif from the statse.webtrendslive.com site.

Tuesday, June 17, 2008

TDD Tips: Create Custom NUnit Categories

In my recent post about test naming conventions and guidelines, I mentioned that you should annotate tests for third-party and external dependencies with category attributes and limit the number of categories that you create. This post will show basic usage of categories, will explain some of the reasoning behind limiting the number of categories. I'll also show how you can create your own categories with NUnit 2.4.x.

Although it's possible to annotate all of your tests with categories, they're really only useful for marking sensitive tests, typically around logical boundaries in your application. Some of the typical categories that I mark my tests with:

  • Database: Tests that require a database to execute. You might want to exclude these tests when you're working remotely or isolate these tests if you need to validate a database deployment for an environment.
  • Integration: Tests that interact with external components you don't have much control over, such as web-services or other infrastructure.
  • Web: Tests that perform regression tests on the visual aspect of a web-site. These tests tend to be very time consuming or require special configuration, so being able to exclude them until they're required can be a big help. Often I run these type of tests when the build server kicks off a build.

Usage

Using categories are very straight forward. Here's an example of a test that is marked with a "Database" category


namespace Code
{
[TestFixture]
public class AdoOrderManagementProvider
{
[Test,Category("Database")]
public void CanRetrieveOrderById()
{
// database code here
}
}
}

Challenges with Categories

One problem I've found with using categories is that category names can be difficult to keep consistent in large teams, mainly because the category name is a literal value that is passed to the attribute constructor. In large teams, you either end up with several categories with different spellings, or the unclear intent of the categories becomes an obstacle which prevents developer adoption.

Fortunately, since NUnit 2.4.x, it's possible to create your own custom categories by deriving from the CategoryAttribute class. (In previous releases, the CategoryAttribute class was sealed.) Creating your own custom categories as concrete classes allows the solution architect to clearly express the intent of the testing strategy, and relieves the developer of spelling mistakes. As an added bonus, you get Intellisense support (through Xml Documentation syntax), ability to identify usages and the ability to refactor the category much more effectively than a literal value.

Here's the code for a custom database category, and the above example modified to take advantage of it:


using NUnit.Framework;

namespace NUnit.Framework
{
/// <summary>
/// This test, fixture or assembly has a direct dependency to a database.
/// </summary>
[AttributeUsage(AttributeTarget.Class | AttributeTarget.Method | AttributeTarget.Assembly, AllowMultiple = false)]
public class RequiresDatabaseAttribute : CategoryAttribute
{
public RequiresDatabaseCategoryAttribute() : base("Database")
{}
}
}

namespace Code
{
[TestFixture]
public class AdoOrderManagementProvider
{
[Test, RequiresDatabase]
public void CanRetrieveOrderById()
{
// etc...
}
}
}

It's important to point out that categories can be applied per Test, per Fixture or even for the entire Assembly, so you have lots of options in terms of the level of granularity.

Filtering Tests using Categories

The real advantage to using categories is that you can filter which tests should be included or excluded when the tests are run.

Filtering Categories within Nunit-Gui.exe

To actively include/exclude tests by category in the GUI:

  1. Click on the Categories tab in the top left
  2. Select the categories you wish to include/exclude, then click the Add button.
  3. If excluding tests, check the "exclude these categories" checkbox.

Filtering Categories in NUnit 2.4.7.

Filtering Categories with Nunit-Console.exe

To include/exclude tests by category from the command line use either the /include:<category-name> or /exclude:<category-name> parameters. It's possible to provide a list of categories by using a comma delimiter.

Example of running all tests within assemblyName.dll except for tests marked as Database or Web.:

nunit-console assemblyName.dll /exclude:Database,Web

Example of running only tests marked with the Database category:

nunit-console assemblyName.dll /include:Database
Note: The name of the category is case-sensitive.

Code Available

I'm pleased to announce that I've setup a repository using Google Project hosting. I'll be posting downloadable code samples. I've created a few simple NUnit categories based on the examples above that you can download and use for your projects:

Happy testing!

submit to reddit

Wednesday, June 04, 2008

TDD Tips: Test Naming Conventions & Guidelines

The idea behind test driven development is that you are writing the test first. Since all code must reside in a method, the very first step before you can write any code, is to name the test. If you're new to TDD, you'll find this to be a very difficult thing to do. Don't let this discourage you, I'd go so far to say that out of all the tasks a developer must accomplish, finding names for things is perhaps the most difficult. W.H. Auden's statement show's that this "meta" thing transcends development:

Proper names are poetry in the raw. Like all poetry they are untranslatable. ~W.H. Auden

This begs a question that comes up frequently for new TDD developers starting out as well as experienced developers during code review: "Is there a naming convention or guidelines for unit tests?" Some believe it to be a black art, but I think it's more like acquiring a rhythm and following along. Once you've got the rhythm it gets easier.

Prior to diving into the guidelines, let's clear up some basic vocabulary:

  • Target / Subject: I often use the term "Target" or "Subject" to refer to the piece of functionality that I'm testing.
  • Fixture: Synonymous with "TestFixture", a fixture is a class that contains a set of related tests. Fixtures are classes that have been decorated with the [TestFixture] attribute.
  • Suite: Test Suites are an older style of organizing tests. They're specialized fixtures that programmatically define which Fixtures or Tests to run. NUnit supports them for backward compatibility by using the [TestSuite] attribute. Since NUnit dynamically finds all tests with the [TestFixture] attribute, they're not as popular these days.
  • Test: You may have noticed that I capitalize Test in all my entries. Tests are methods within the Fixture that are decorated with the [Test] attribute and contain code that validates the functionality of our target.
  • Setup/TearDown: Test Fixtures can designate a special piece of code to run before every Test within that Fixture. That method is decorated with the [Setup] attribute. Likewise, a method with the [TearDown] attribute is called at the end of every test within a fixture.
  • Fixture Setup/Fixture TearDown: Similar to constructors and finalizers, methods with the [TestFixtureSetup] or [TestFixtureTearDown] attributes execute before and after the Fixture executes. These methods happen before [Setup] and after [TearDown].
  • Category : The [Category] attribute when applied to a method associates the Test within a user-defined category.
  • Ignore: Tests with the [Ignore] attribute are skipped over when the Tests are run.
  • Explicit: Tests with the [Explicit] attribute won't run unless you manually run them.

The following are some suggestions I've adopted or recommended to others from past projects. Feel free to take 'em at face value, or leave a comment if you have some to add:

Fixtures

DO: Name Fixtures consistently
TestFixtures should follow a consistent naming convention to make tests easier to find. Choose a naming convention such as <TargetType>Fixture or <TargetType>Test and stick to it.

DO: Mimic namespaces of Target Code
To help keep your tests organized, use the same folders and namespace structures as your target assembly. This will help you locate tests for target types and vice versa. Since most Test runners group Tests by their namespace, it's really easy to run all tests for a specific namespace by selecting by the container folder -- which is great for regression testing an area of code. I've got another post which talks about how to structure your Test namespaces.

DO: Name Setup/TearDown methods consistenty
When naming your fixture setup and teardown methods, you really should pick a style for these methods and stick with it. Personally, I can't find any reason why you would deviate from naming these methods FixtureSetup, FixtureTearDown, Setup, and TearDown as these provide clear names. By following a standard TestFixtures structure you can cut down some of the visual noise, make tests easier to read and produce more maintainable tests across multiple developers.

CONSIDER: Separating your Tests from your Production Code
As a general rule, you should try to separate your tests from your production code. If you have a requirement where you want to test in production or verify at the client's side, you can accomplish this simply by bundling the test library with your release. Still, every project is different, and tests won't necessarily impede production other than bloating up your assembly. Separate when needed, and use your gut to tell you when you should.

CONSIDER: Deriving common Fixtures from a base Fixture
In scenarios where you are testing sets of common classes or when tests share a great deal of duplication, consider creating a base TestFixture that your Fixtures can inherit.

CONSIDER: Using Categories instead of Suites or Specialized Tests
Although Suites can be used to organize Tests of similar functionality together, Categories are the preferred method. Suites represent significant developer overhead and maintenance. Likewise, creating specialized folders to contain tests (ie "Database Tests") also creates additional effort as tests for a particular Type become spread over the test library. Categories offer a unique advantage in the UI and at the command-line that allows you to specify which categories should be included or excluded from execution. For example, you could execute only "Stateful" tests against an environment to validate a database deployment.

CONSIDER: Splitting Test Libraries into Multiple Assemblies
From past experience, projects go to lengths to separate tests from code but don't place a lot of emphasis on how to structure Test assemblies. Often, a single Test library is created, which is suitable for most projects. However, for large scale projects that can have hundreds of tests this approach can get difficult to manage. I'm not suggesting that you should religiously enforce test structure, but there may be logical motivators to divide your test assemblies into smaller units, such as grouping tests with third-party dependencies or as an alternative for using Categories. Again, separate when needed, and use your gut to tell you when you should. (You can always go back)

AVOID: Empty Setup methods
As a best practice, you should only write the methods that you need today. Adding methods for future purposes only adds visual noise for maintenance purposes. The exception to this is when you are creating a base Fixture that contains empty methods that will be overridden by derived classes.

Tests

DO: Name Tests after Functionality
The test name should match a specific unit of functionality for the target type being tested. Some key questions you may want to ask yourself: "what is the responsibility of this class?" "What does this class need to do?" Think in terms of action words. Well written test names should provide guidance when the test fails. For example, a test with the name CanDetermineAuthenticatedState provides more direction about how authentication states are examined than Login.

DO: Document your Tests
You can't assume that all of your tests will be intuitive for everyone who reviews them. Most tests require special knowledge about the functionality your testing, so a little documentation to explain what the test is doing is helpful. Using XML Documentation syntax might be overkill, but a few comments here and there are often just the right amount to help the next person understand what you need to test and how your test approaches demonstrates that functionality.

CONSIDER: Use "Cannot" Prefix for Expected Exceptions
Since Exceptions are typically thrown when your application is a performing something it wasn't designed to do, prefix "Cannot" to tests that are decorated with the [ExpectedException] attribute. Some examples: CannotAcceptNullArguments, CannotRetrieveInvalidRecord.

I would consider this a "DO" recommendation, but this a personal preference. I can't think of scenarios where this isn't the case, so this one is up for debate.

CONSIDER: Using prefixes for Different Scenarios
If your application has features that differ slightly for application roles, it's likely that your test names will overlap. Some teams have adopted a For<Scenario> syntax (CanGetPreferencesForAnonymousUser). Other teams have adopted an underscore prefix _<Scenario> (AnonymousUser_CanGetPreferences).

AVOID: Ignore Attributes with no explanation
Tests that are marked with the Ignore attribute should include a reason for why this test has been disabled. Eventually, you'll want to circle back on these tests and either fix them or alter them so that they can be used. But without an explaination, the next person will have to do a lot of investigative work to figure out that reason. In my experience, most tests with the Ignore attribute are never fixed.

AVOID: Naming Tests after Implementation
If you find that your tests are named after the methods within your classes, that's a code smell that you're testing your implementation instead of your functionality. If you changed your method name, would the test name still make sense?

AVOID: Using underscores as word-separators
I've seen tests that use_underscores_as_word_separators_for_readability, which is so-o-o 1960. PascalCase should suffice. Imagine all the time you save not holding down the shift key.

AVOID: Unclear Test Names
Sometimes we create tests for bugs that are caught late in the development cycle, or tests to demonstrate requirements based on lengthy requirements documentation. As these are usually pretty important tests (especially for bugs that creep back in), it's important to avoid giving them vague test names that represent a some external requirement like FixForBug133 or TestCase21.

Categories

DO: Limit the number of Categories
Using Categories is a powerful way to dynamically separate your tests at runtime, however their effectiveness is diminished when developers are unsure which Category to use.

CONSIDER: Defining Custom Category Attributes
As Categories are sensitive to case and spelling, you might want to consider creating your own Category attributes by deriving from CategoryAttribute. UPDATE: Read more about custom NUnit Categories.

Well, that's all for now. Are you doing things differently, or did I miss something? Feel free to leave a comment.

Updates:

  • 6/18/08 - Added links to custom NUnit Categories

submit to reddit