Running tests in DesktopBuild

This post is inspired by a problem we had in our TFS2008 build environment, I am not sure if it applies to the 2005 version of MSBuild.

The problem

Say you are running tests in your team builds, and you have configured your build so they can run on a Developer desktop as well as on a build server. By default, using Microsoft’s targets, tests are not run during a Desktop Build. This means that a build can pass when run locally but fail, or give partial success on the server, due to test errors. We had a scenario where some tests were “Not runnable” in our Test Container. Which resulted in failed builds.

The reason is that MSTest returns the error code 1 if there are tests that are not runnable.

To test the behaviour locally we wanted our tests to run during the Desktop Build, so we could have a faster debug cycle when working with this problem.

Solution:

By looking in the Microsoft.TeamFoundation.Build.targets file, we see what dependencies the test targets have.

Here is a list starting from the Test target. If a target has a dependency it is located above the target. If a target results in the execution of MSBuild on different targets it is represented by the content in the column to the right column.

<Target Name=”ResolveTestFilesForEndToEndIteration” Condition=” ‘$(IsDesktopBuild)’ != ‘true’ “>
	<WorkspaceItemConverterTask Condition=” ‘@(MetaDataFile)’ != ” ” TeamFoundationServerUrl=”$(TeamFoundationServerUrl)” BuildUri=”$(BuildUri)” WorkspaceName=”$(WorkspaceName)” WorkspaceOwner=”$(WorkspaceOwner)” ServerItems=”@(MetaDataFile)”>
		<Output TaskParameter=”LocalItems” ItemName=”LocalMetaDataFile” />
	</WorkspaceItemConverterTask>
	<WorkspaceItemConverterTask Condition=” ‘@(TestContainer)’ != ” ” TeamFoundationServerUrl=”$(TeamFoundationServerUrl)” BuildUri=”$(BuildUri)” WorkspaceName=”$(WorkspaceName)” WorkspaceOwner=”$(WorkspaceOwner)” ServerItems=”@(TestContainer)”>
		<Output TaskParameter=”LocalItems” ItemName=”TempTestContainer” />
    </WorkspaceItemConverterTask>
	<!– Expand TempTestContainer wild cards to create LocalTestContainer item that will be used later (this will replace * and ? with matching file paths)–>
    <CreateItem Include=”@(TempTestContainer)”
	<Output TaskParameter=”Include” ItemName=”LocalTestContainer”/>
</CreateItem>

We can make our own version of this and add it to our TFSBuild.proj file that runs on Desktop Builds. What we do is simply pass on the TestContainer, without using the WorkspaceItemConverterTask.

<Target Name=”ResolveTestFilesForEndToEndIteration” Condition=” ‘$(IsDesktopBuild)’ == ‘true’ “>
	<!– Expand TempTestContainer wild cards to create LocalTestContainer item that will be used later (this will replace * and ? with matching file paths)–>
	<CreateItem Include=”@(TestContainer)”>
		<Output TaskParameter=”Include” ItemName=”LocalTestContainer”/>
	</CreateItem>
</Target>

With this target included in our TFSBuild.proj file, the build runs the tests both locally and on the server.

For our project, we have added this target to our common targets file that all builds refer to. So that all projects run tests when run on as a Desktop Build.