A common scenario I encounter is when a client has some existing assets in TFS or VSTS and they want a clean slate, to consolidate backlogs or to use a new Process Template.  More recently it is because they want to move their backlog to Visual Studio Team Services (formerly Visual Studio Online)

If you want to migrate source code history then you’re looking at something like the TFS Integration Tools for on premise TFS or the OpsHub Visual Studio Online Migration Utility for VSTS.  They will also migrate Work Items and their links. 

There are other open source tools for migrating Work Items out there such as Total TFS Migration but they are not always maintained and there are some commercial tools which can be expensive.

If the situation is right then I often find Excel is the simplest mechanism to copy Work Items from one Team Project to another.  I have also used the same technique to successfully move items from Jira, Trello and other systems.

However, using an Excel input list is not a perfect solution on its own.  Off the top of my head, you’re going to have to consider:

• New Work Item IDs
• Work Item Links
• Creation State
• Hyperlinks
• Attachments
• Lost Work Item history
• Discussions
• Test Steps
• Created Date
• Closed Date
• Area Paths
• Iteration Paths

We’ll look at some of those later in this post but let’s get some Work Items copied to begin with.

The simplest scenario is if both source and target projects use the same Process Template so that all your fields match (eg. copy the value of Microsoft.VSTS.Scheduling.Effort from source to target). 

If the templates don’t match then you’ll have to do some planning to understand how the fields match  (for example Microsoft.VSTS.Scheduling.StoryPoints to Microsoft.VSTS.Scheduling.Effort) or fields that might be missing (eg. Microsoft.VSTS.Scheduling.CompletedWork or Microsoft.VSTS.Scheduling.OriginalEstimate).  If the fields are missing then you could consider adding them to the target process template, storing the information somewhere else or simply leave it behind.

Create a Query

First up we need a query to return all the items you want to copy. 

Ask yourself, are you going to copy closed, done or removed Work Items.  Often clients do but don’t realise that it’s not going to give them the historical view that they expect as the Closed Dates will be different.  I’d encourage you to leave them behind and retain the old project for a while.

If you have any links between Work Items that you wish to preserve then it needs to be a Tree Query.  We could hook up links with PowerShell later on (mental note to blog about that later) but I like to use Excel if possible.

A query like this is going to bring back everything in a hierarchical structure regardless of the State or Work Item type. 

Something like this will omit Done or Removed Work Items but just be careful that you don’t lose orphaned items or perhaps an odd situation where you have active child items whose parents have been moved to Done.

You may need to play around with the query to get what you want. 

Importantly, click on Column Options and choose which fields you wish to import.

Then choose your sort order.  Backlog Priority followed by ID is probably a good shout.

Copy your Query to the target project

We have our query to export the data we want but now we need to add that to the target project so that we can import the data.

Open the source project in Visual Studio/Team Explorer and navigate to your query.

Click on Edit Query

And Select File-> Save As…

We want to save our query to a file (*.wiq), somewhere we we can find it.  Open the file in a text editor of your choice (I like Notepad++) or you can do it in Visual Studio as long as you select the Open With drop down.

Edit the Server URL and the Team Project Name in the wiq file and save it again. 

Now open Visual Studio and connect to your target team Project. 

Select File->Open->File…

Navigate to your *.wiq query file.

Select Save As…

and select your new project as the target server. 

The query should now be available in the target Team Project.

Open your Query in Excel

Open Excel, navigate to the Team tab, hit New List and select your query in the source Team Project.

Open another instance of Excel (hold down the Shift key when you click the shortcut) and open the query in the target Team Project.

In the source project we can see the Work Items we are going to migrate.  Note that the Parent/Child hierarchy is represented by multiple Title columns.

This hierarchy is missing on the target query so we need to add as many levels as we need.

On the Team ribbon, select Add Tree Level, in my case I needed three more levels to my hierarchy to represent Epic-Feature-PBI/Bug-Task.

Create new columns

Now we need to create some space where we can store related information that we will not publish to the new team project.  You could add custom fields in TFS to store this information but I like to keep the template as clean as possible.

Right click on the column in Excel and Insert columns where you need them.

Repeat the process to store things like the Old Id, the Old State (all new items will be created as New/To Do etc), Old Area Path and so on.

I like to colour these columns (it’s always orange for some reason) so that it is clear that they will not be published back to TFS and only live on this spreadsheet.

Clicking Choose Columns in the Team ribbon allows you to re-order the columns including the non-TFS ones.  This will make it easier to copy and past in the next section.

Copy source Project data

Begin the process of copying the data from the source sheet to the target sheet.  When we hit Publish on the Team menu, new Work Item IDs will be generated for us in the new project and the Parent-Child links will be created.

Edit the copied data

Now we need to alter some things like the Work Item State.  The TFS Process Template will not allow us to create Work Items in a non-starting state so our items will be New or To Do for example.  We need to filter on the Old States we want to edit.  Click the drop down on the column header and select the first state to edit.

Now we see only the items we need to change.  Change and then fill down the value in the State column (In Progress in the example below) and publish back to TFS.  Repeat the process for all the states returned in your query

You will also need to come back and do this for things like Area Path and Iteration Path if you haven’t already created them in your new project.  Just save this spreadsheet and you can refresh and publish from/to TFS anytime.

Conclusion

So now we have our basic data in our new Team Project along with the Work Item hierarchy.  For many this will be enough but for others, there will be hyperlinks to SharePoint documents, attachments and more that need to be migrated.

Colin Dembovsky has a great blog post - Bulk Migrate Work Item Comments, Links and Attachments – which provides a PowerShell script which is really helpful. 

I want to talk about some of the other issues such as Test Steps and Area Paths in another article but for now it’s time for Christmas.

Cheers,

Richard

There are some great features coming in Team Foundation Server 2015 and they’re already largely available in Visual Studio online. 

My personal favourite is the enhanced Agile tooling in Team Web Access and in particular the improved support for Kanban (swimlanes, explicit policies etc.).  However, I love the new simplified build system and its tight integration with test and release.  Release Management is also ready for the big time with cross platform support and a new web client.

TFS 2015 is due for release on 20th July and there are a lot of reasons you should think about upgrading.  In addition, editable process templates are coming to Visual Studio Online so you might also be thinking about a move to the cloud.

However, do you find yourself running TFS 2010 or TFS 2012 (even 2005 or 2008 if you’re really unlucky) and now you’re one more version behind which can be a problem for supportability?

Maybe you have a heavily customised process template or you’re using Scrum For Team System and you want to change it as part of the upgrade?

If you need some help and advice on the options open to you then drop us a line. 

We can help assess your situation and leave it to you from there or take care of the whole upgrade process and follow up with training and coaching.  We’ll tailor any engagement to suit your requirements.

Richard

I was working on a client’s computer yesterday to make some Process Template changes on TFS 2013.  I fired up a command prompt, typed witadmin /? to see if it was on the path an off I went.

I exported a Work Item definition (witadmin exportwitd), added a field and tried to import the changes (witadmin importwitd).  I then received a couple of schema validation errors along the lines of:

TF212019: Work item tracking schema validation error at row x, column y: The HideControlBorders attribute is not declared.

TF212019: Work item tracking schema validation error at row x, column y: The HideReadOnlyEmptyFields attribute is not declared.

I checked my changes, and checked that the attributes were present.

<FORM>
     <Layout HideControlBorders="true" HideReadOnlyEmptyFields="true">
     …

It wasn’t one I’d seen before, the changes imported correctly on my own VM and there were no hits on a quick web search (hence I’m writing this blog!)

I know those attributes were added in TFS 2012 and that should have given me a clue as to what was causing the problem but instead I wasted time investigating other causes.

It wasn’t until I made a change to the categories.xml that it became obvious.  Again, export was fine but an import gave a more useful error message about my version of witadmin.

Doh!  I quickly checked and the client had both VS2013 and VS2010 installed so the wrong version of witadmin.exe was on his path.  It wasn’t my machine to mess with so I just did a quick cd to:

C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE

I hit the same witadmin importwitd and everything was fine.

Cheers,

Richard

Final post for tonight and then I might have a little nap on the train.

On another recently created project utilising the custom Team field to scope Work Items to TFS teams rather than the Area Path I encountered another minor issue.

I created a Test Plan using Microsoft Test Manager and then clicked the Open in Web Access button

I received the following error in Team Web Access:

The test plan with id XX does not exist or it’s area path is not owned by the default project team.  Include it in the default project team’s owned areas and try again.

The error is a little cryptic as I hadn’t even started adding area paths, much less locking them down.  It was in fact, another symptom of using the custom Team field,

I hadn’t implemented the template customisation and so the Work Item definition was missing the following on the Team field:

<DEFAULT from="value" value="Unassigned" />

TFS 2013 Update 3 requires that you add your Team field to the new Test Plan Work item.  You don’t have to expose the field on the UI but it must have a default value or you will hit issues in Team Web Access when accessing those Test Plans that do not have the field populated.

Easy to resolve –

• Run a query into Excel that brings back all the Test Plan Work Items
• Add the Team field to the list of columns
• Set the Team value for the Test Plans to Unassigned (or whatever you use)
• Use witadmin.exe or the Process Template Editor to update your Work Item definitions to have a default Team value

Time for a snooze.

Cheers,

Richard

My blog posts are like really terrible buses, you wait 8 weeks for one and now I’ve finally got a little time (3 hours on a train home from the midlands) so 2 or 3 are going to arrive at once.

A process template customisation I do regularly for customers is adding a Team field to scope Work Items to teams in TFS 2012 and TFS 2013.  Microsoft have detailed the process here so it’s not a particularly controversial thing to do and I believe it provides a better experience when you have multiple teams working against a central backlog.

I blogged recently about how to use PowerShell to automate the process of adding a team field.

I performed the customisation for three companies in the last couple of months and bumped into a some issues with Team Web Access in particular.  Even though adding a Team field is very much a valid thing to do, there can be a couple of wrinkles to iron out.

The first issue came as a bit of a shock.  The background to the problem is this - a customer has been happily using TFS 2013 on a pathfinder project with a single team.  Things have gone well and so they wish to roll out the tool to about 5 other teams.

We worked together to create a slightly customised process template which added some further hierarchy to the standard Visual Studio Scrum 2013 process template.  Without thinking, I started work using a Visual Studio Scrum 2013.3 template and when I tried to upload it to the customer’s server we had some errors as Update 3 added all sorts of test related artefacts that weren’t available with the customer’s TFS 2013 RTM version. 

That wasn’t the issue, however and I quickly made the changes to an RTM version of the VS Scrum template and uploaded it successfully.  We then created a central project to house the company’s portfolio backlog (I must add that to my things to blog about!).

We were to resume working on the new project a few days later but before then I received a frantic email from the pathfinder team saying that whenever they tried to edit an existing Work Item using the Team Web Access client they received the following error:

Team Foundation Error

The server operation failed, (Value cannot be null. Parameter name:key).

I quizzed the team on whether they had changed anything on the pathfinder project as templates are like cookie cutters, once you cut out the cookie, it doesn’t matter what you do to the cutter, it doesn’t effect the cookies you produced previously.

I went onsite and pulled down the processconfiguration.xml, categories.xml and the Work Item definitions for the pathfinder project.  Nothing had changed!

Some experimentation confirmed that uploading a new process template configured to use a custom team field had broken Team Web Access for an existing project.  Clearly that shouldn’t happen and although I we could have battled through and fixed it we took the decision to update the TFS 2013 RTM server (without knowing for sure that Update 2/3/4 had fixed the problem).  We took the decision to go with Update 2 as we weren’t quite ready for the new test artefacts and I’d recently been badly burned by an issue with Update 3 massively inflating the collection database (another thing to add to my list of blog posts to write).

The upgrade to Update 2 only caused us a short period of downtime and it was something we were planning on doing soon anyway.  The problem was resolved immediately so it was obviously a known bug which was addressed in the first update for TFS 2013 (there was no update 1 for TFS 2013).

A similar but not identical issue is discussed in the MSDN forum post here.

There are still 90 minutes left of my train journey so I may as well write up the next instalment.

Cheers,

Richard