A few years before I left Microsoft UK to join RippleRock, I remember having a conversation with one of my future colleagues about RippleRock’s value proposition and their response was simply this:

“We will make your Agile project successful”

Now, of course they went on to caveat that this requires buy-in from the business among other things but RippleRock’s specialism in both Lean/Agile process and ALM/DevOps tooling give them a unique position among the Microsoft UK Partners.

Since I joined the team, RippleRock have continued to deliver on that statement but there is one thing that comes up project after project and that is the difficulty and inaccuracy teams encounter with gaining usable insight into their development process . On top of that, most organisations insist on reporting on metrics that will only drive the wrong behaviour and cause dissatisfaction in the team.

Some examples include:

• How accurate are the team’s initial estimates?
• How many bugs have the testers raised?
• How many lines of code have been written?
• Comparing velocity across teams
• Cost per team/Story Point

This type of behaviour and the difficulty in authoring reporting (initially in TFS/VSTS but also in Jira etc.) caused RippleRock to develop SenseAdapt.  The name obviously comes from the tool’s aim which is to allow teams to ‘sense’ what is going on in their project and ‘adapt’ their process in order to improve things.

Some examples of what SenseAdapt is visualising for your team:

• When will this project complete?
  • Based on real historical data and using Monte Carlo simulation to predict the likely range of completion dates.
• How healthy is our backlog?
  • Are we working on the right things?  Have we given the team all the information they need to do their jobs?
• How much ‘sludge’ is in our backlog?
  • Typically, there is lots.  This is work that never makes its way to the top of the backlog and will never be addressed, it just sits there, growing and demoralising the team
• Where are the bottlenecks in our process?
  • How can we improve the flow of value to users

^{Figure 1: Currently available charts in SenseAdapt}

Our clients have been using SenseAdapt for five months and the insights that it brings to teams and management is leading to real change in both process and project scope as they rely less on the current subjective reporting and groupthink.

Recently we came up with what I think is the perfect description of what RippleRock are enabling with SenseAdapt:

Process Telemetry

Whilst we use tools such as Application Insights and Dynatrace to monitor a running application or website, SenseAdapt monitors your development process and helps you to see and then take action to improve the system.

If you are interested in a demo or a trial of SenseAdapt or want to discuss what it could offer you then contact us.

If you are UK based then come along to our free event at Microsoft Campus in Reading on 6^th May where we will talk about all this and much more.

More details and signup here:

Engaging the Business with Team Foundation Server: From Idea to Live – Microsoft Campus, Reading (Friday 6th May 2016)

More information on SenseAdapt available on http://www.senseadapt.com

Cheers,

Richard

Tweet.

RippleRock are developing an awesome tool for providing realtime, actionable visualisations of your project data.  If you’re wondering why your backlog is such a mess, want to understand how you could improve your Agile/Lean development process or simply want to predict when your project will end, then you should check out SenseAdapt.

SenseAdapt works with Team Foundation Server and Visual Studio Team Services (keep an eye out for a preview on the new VSTS Marketplace soon) and we’re experimenting with a Jira version too.

To develop SenseAdapt, we use our own project data, so SenseAdapt is helping us to improve and adapt the process we use to develop SenseAdapt!

The problem is that we need real data in both VSTS and TFS and as we recently consolidated a VSTS project into our on-premise TFS 2015, we wanted a one-way sync of Work Items between the two systems.

There are options of course:

• Commercial Product like TaskTop or OpsHub
• Roll our own, with a TFS server side plug-in (much the way we do with TFSRippler)
• A free tool such as the TFS Integration Tools.

I wanted something implemented quickly and cheaply so that’s why yesterday I installed the TFS Integration Tools.  If you have ever used TIT (they really should have thought about that acronym) then you’ll know that they are……er, interesting?

Last released in March 2012, they know nothing of TFS 2013/2015 and VSO/VSTS was the TFS Service Preview back then.  My installation was on Windows Server 2012 R2 with SQL Server 2014 Standard SP1 and TFS 2015 RTM (No update 1 on this instance for testing purposes)

I knew that I needed to install the TFS 2012 Team Explorer so that the Integration Tools could talk to my server using the older object model.

I also knew there was a problem with this setup that needed a little registry change.  The error you will see is:

“This tool requires the TFS client object model.  Please install Team Explorer 2008, Team Explorer 2010, or Team Explorer Dev11 and run setup again”

An old blog post by Willy Peter Schaub shows that the easiest way to fix this is to paste the following into a .txt file, rename to .reg, and double click.

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\11.0\InstalledProducts\Team System Tools for Developers]

@="#101"

"LogoID"="#100"

"Package"="{97d9322b-672f-42ab-b3cb-ca27aaedf09d}"

 "ProductDetails"="#102"

"UseVsProductID"=dword:00000001

I was off and running now so I kicked off the install that I’ve done many times in the past and selected to include the TFS Integration Service

For the database server name, i just put a ‘.’ as I was going to use the default instance (SQL 2014 Standard)

Unsurprisingly, something went wrong but I aborted the install without really paying attention, assuming I messed something up.

So I started the installation again, paying more attention this time but I received a different error:

Database Error: Cannot read TFS Integration Tools database version number

It turns out that the failed installation had created the database Tfs_IntegrationPlatform but had not cleaned it up again during rollback.  That helpful message meant that I needed to go into SQL Management Studio and delete the database.

We were off again and everything seemed good until the error message that I received originally showed up again.

Error -2147217900: failed to execute SQL string, error detail: Valid values of the database compatibility level are 100, 110, or 120., SQL key: CreateDatabaseScript SQL string: EXECUTE sp_dbcmptlevel [Tfs_IntegrationPlatform], 90;

This one was trickier.  There was an ooooold forum post by Bill Essary at Microsoft saying that they were aware of the issue but didn’t have a fix as yet.  A few others suggested that the problem went away when they retried the install.  After the 3rd time, I accepted it wasn’t going away.

Making an educated guess, i assumed that the integration tools were not compatible with SQL 2014 so I decided to install a named instance of SQL 2012 Standard SP3 on the same server.

Passing in .\INTEGRATION as the database server\instance meant we were finally installed!

Onto the next problem

I created a new configuration and selected Team Foundation Server\WorkItemTracking.xml as my template.

I didn’t bother reading the docs as I was still sceptical that this was going to work so I just chose Custom workflow, ContinuousAutomatic frequency, Unidirectional direction.

Next, I chose my on-premise TFS 2015 server as the Left Source and my VSTS account as the Right Source.

I had forgotten that the account running the Sync/Migration had to be in the Service Accounts Group.  On-premise you have to use tfssecurity.exe to add your account to that group but I suddenly thought, “there’s no way VSTS will let me do that”, so I expected it to be game over.

I tried:

tfssecurity.exe /g+ “Team Foundation Service Accounts” n:my@emailaccount.com /server:https://myaccount.visualstudio.com

and unsurprisingly got

The requested operation is not allowed

Kind of obvious as VSTS is not going to allow me to be a Team Foundation Admin or Service account so I had one last stab:

tfssecurity.exe /g+ “Project Collection Service Accounts” n:my@emailaccount.com /collection:https://myaccount.visualstudio.com/defaultcollection

Amazingly, that was successful

I repeated the process for my on-premise server and I was off.

Now, the tools are installed and running but as usual, it has thrown a hissy fit on the first pass.

 

Application Error: 0 : A failure occurred while trying to load the C:\Program Files (x86)\Microsoft Team Foundation Server Integration Tools\Plugins\Tfs2008WITAdapter.dll Plugin:
System.Reflection.ReflectionTypeLoadException: Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information.

and

Application Error: 0 : A failure occurred while trying to load the C:\Program Files (x86)\Microsoft Team Foundation Server Integration Tools\Plugins\Tfs2010WITAdapter.dll Plugin:
System.Reflection.ReflectionTypeLoadException: Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information.

and

Application Critical: 0 : [27/01/2016 19:07:38] System.InvalidOperationException: Specified element is already the logical child of another element. Disconnect it first.

After all that, those errors looked like they could be terminal but I found an article that states that removing any files with 2008 or 2010 in the name from the following folder:

C:\Program Files (x86)\Microsoft Team Foundation Server Integration Tools\Plugins

will allow the Integration Tools to progress.

Great source for all things Integration Platform: TOC: TFS Integration Tools Blog Posts and Reference Sites

And we’re off and running with 1-way sync between TFS 2015 and Visual Studio Team Services!

Cheers,

Richard

Tweet.

One of my clients recently moved to Visual Studio Team Services (formerly Visual Studio Online/VSO) and decided to share an account already in use by another part of the company. 

Their developers were mainly licensed with MSDN and so were able to hop straight on there and start working but the ScrumMasters, Product Owners and testers were not and they wanted to know how they should license those non-developer users.

I realised that a lot had changed since the last time I looked at the VSO licensing so it’s worth a quick blog in case others were bumping into the same issue

Free Users

When you create a Visual Studio Team Services account (myaccount.visualstudio.com) you get 5 free users.  This means that if you’re creating the account for some personal dev project, you could invite 4 friends to contribute and nobody needs any licenses for anything!  Go and grab Visual Studio Code or Visual Studio Community for free and get stuck in to coding/building.  Grab the Office Plug-in and your “Product Owner” can hook the backlog and other queries into Excel (you need an office license obviously!).

The free users are “Basic” users - we’ll talk more about this later.

My client had joined an existing account so when they navigated to:

https://myaccount.visualstudio.com/_user

all the free users had been assigned so we needed to add some other types.

MSDN Users

Anyone with an MSDN subscription (Enterprise\Professional\Test Professional\Platforms) has an included Client Access License (CAL) that can be used with on-premise Team Foundation Server and/or cloud hosted Visual Studio Team Services. 

You can be added to any number of VSTS accounts at the Basic level  so I can be on my personal account, the RippleRock account and client accounts)

Stakeholders

Free Stakeholder licenses are great, as the name suggests they were added to allow anyone who has an interest or a say in the project to view project information without the need to buy them a license.  If they simply want to add some things to the bottom of the backlog for consideration by the Team/Product Owner, or view project progress, then great, hook up as many as makes sense and away they go.

However, many teams will think, “my ScrumMaster doesn’t need a license as they’re not writing code”, or “my PO doesn’t need a license as they only want to work with the backlog”, but that generally isn’t the case.

There is a good comparison of Stakeholders and Basic Users here:

Visual Studio Team Services Feature Matrix

In summary, Stakeholders:

• Can view, create, edit and query (most) Work Items via the web client
• Cannot access Work Item Queries from Excel
• Have limited functionality on backlogs and boards (crucially the cannot re-order items which is pretty critical for PO/SM)
• Have no access to version control/builds
• Cannot be project administrators

Additional Users

Prior to VSO/VSTS, if you needed to license a non-MSDN user for TFS, you simply bought a Client Access License (CAL) for TFS. 

CALs were not very expensive but they could be bought with or without Software Assurance and one problem was that users often bought the license only (without SA) which meant it was tied to a specific version of TFS.  This meant that when the team wanted to upgrade to the latest version of TFS, it could become a more costly procedure as new CALs had to be purchased.

With VSTS, you can still buy a CAL but it is usually significantly cheaper and more flexible to add users through VSTS (which also allows them to access both on-premise and cloud hosted) and pay monthly for what you need.

You need to set up payment on your account and choose how many users you need.

Pay for users accessing your account

Pricing f0r additional users is pretty reasonable.  Remember, the first 5 users are free, so if you need 10 users it will cost about £20 per month (each user over 5 is approximately £4, or more accurately $6).  If you need 15 users then it’s about £48 per month in total.

First 5 users: FREE
Users 6 through 10: $6 each
Users 11 through 100: $8 each
Users 101 through 1000: $4 each
Users 1001 and above: $2 each

At the time of writing, if you have an Enterprise Agreement, then all additional users are only $4 per month which makes it about 23 Basic users for only 50 quid a month!

For more details and a handy calculator, see:

 Visual Studio Team Services Pricing

That covers Basic users but what about VSO Professional or VSO Advanced licenses?

In short, they’re gone!

Brian Harry covered this in his blog in September:

September pricing and licensing changes

Now, you simply provision a Basic user and then purchase add-ons for any additional functionality that they need.  At the time of writing this additional functionality is limited to Test Manager (that used to be included with the Advanced license) but the new marketplace will mean that much more will be available in the future.

Test Manager

While we’re at it, where’s Team Explorer 2015?

It’s gone too!  The main reason people installed Team Explorer was to get the incredibly useful Excel integration but there will be no Team Explorer for Visual Studio 2015.

You can install the free Visual Studio Express or Visual Studio Community to get your Team Explorer or you can continue to use Team Explorer for Microsoft Visual Studio 2013 which will happily connect to VSTS/TFS 2015.

If you simply want the Excel integration then you can grab the new TFS Office plug-in

More information: TFS Office Integration Installer

Download: Microsoft Team Foundation Server Office Integration 2015 Update 1

Resources

As always, the Visual Studio 2015 Licensing White Paper usually has any information you need and is also handy if you’re having trouble sleeping.

Cheers,

Richard

Tweet.

*Update April 2016 – If you are interested in this area and based in the UK then register for our free event at Microsoft Campus in Reading on 6th May*

About 18 months ago I wrote a blog post on TFS Reporting and it has been one of the most popular on my blog for some time.  I think that is mainly because reporting in TFS is so difficult but many teams realise the importance of accessing the data that it contains. 

Although 90% of that post is still valid in the new world of Team Foundation Server 2015 (Update 1) and Visual Studio Team Services, there have been some fairly significant enhancements and also my approach to reporting have changed too.

Visual Studio Team Services Vs. Team Foundation Server

At the time of my first post on this subject, the cloud-hosted version of TFS had just been renamed to Visual Studio Online (VSO) from Team Foundation Service Preview and reporting was one of the few missing features as it did not have a data warehouse or SQL Server Reporting Services. 

However, at the end of 2015, VSO has been renamed again to Visual Studio Team Services (VSTS) and those missing features have been added (custom Process Templates), become obsolete (SharePoint Integration) or in some ways surpassed the current on-premise version (PowerBI reporting, Release Management).

This is because VSTS is evolving rapidly and is updated monthly whereas we typically see on-premise updates quarterly and it could take IT many more months to plan applying the update.  At RippleRock, we run all our internal projects on VSTS and for me it is fast becoming the obvious choice over on-premise as long as you can get over the idea of your project data and source code being stored in the cloud.

So what has changed in the last 18 months?

Charting

Admittedly, Charting hasn’t changed massively but a historical set of charts was introduced which made the feature much more useful.  Take a look at the Trend Charts in addition to the Snapshot charts

I am not going to repeat the details of creating a chart as they are well documented on MSDN:TFS2015/VSTS Charts.  One thing I will note is that you can only create a chart from a Flat Query which can be a little limiting.  Pay close attention to the included Columns and Sort Order of your query too, as they will have an impact on what is available to you in the chart options.

 

Dashboards

In TFS 2013 each team had a home page where Team Administrators could pin queries and charts specific to that team.  However, these homepages could quickly become cluttered and had to serve multiple purposes making it difficult to filter the data you wanted to see.  With TFS 2015 Update 1 (and VSTS since October 2015) this has changed to allow users to create multiple dashboards per team.  So you could easily have a bugs dashboard, stakeholder dashboard etc.

Again this is well documented on MSDN: TFS2015U1/VSTS Dashboards.  Just click the green ‘+’ beside the Overview dashboard and start creating the boards you need.

Each board can have a number of configurable widgets to allow you to only show information relevant to the board.  At the time of writing there are about 15 widgets that can be added to a dashboard by clicking the ‘+’ on a green circle at the bottom of the dashboard.

What is exciting for RippleRock is that you will soon be able to write your own widgets and we will make our SenseAdapt (see below) charts available to your dashboards to give realtime actionable metrics.

Power BI

Power BI is probably the biggest development since the last post and typically the feature was first announced shortly after I published the blog making it almost instantly out of date.

To be fair, at that stage there wasn’t a lot in there, really just some nice metrics for Git users in TFS -  No TFVC, No Work Items, No Test Results.  A lot has been added since then and there is more to come!

Once you’ve signed up, it is pretty easy to connect Power BI to your VSTS account (just remember not to add the .visualstudio.com when you enter your account name).

 

When choosing your project(s), be careful to spell it correctly as it doesn’t appear to check if the project exists and you’ll just get blank data.

Importantly data is not live, this is similar to the OLAP cube in TFS which was refreshed every 2 hours by default.  The last time I checked, PowerBI updated data every 24 hours but this is configurable.

Now you can interact with data in a number of ways.  There are the pre-created reports, natural language queries (how many bugs were created this week?) and then you can create your own reports much like you do with Excel and SSRS.

On premise support for PowerBI is not there yet.

However, it can be done by pointing the cloud hosted PowerBI at your on-premise data warehouse.  Obviously this adds some complexity around access to a server behind the firewall and in my eyes doesn’t get beyond the problems that we see with 3rd Party BI Tools.  I think this will become the main reporting option for TFS but for me, it’s not fully baked yet.

SenseAdapt

SenseAdapt is a custom reporting tool for TFS that was born out of both the frustration at the difficulty of visualising data from TFS and the realisation that all too often the data in the system was poor.  We needed something to allow us to understand how teams were using TFS and to surface issues that could be tackled.

My colleague, Bazil is our Product Owner and has a great blog post on SenseAdapt

SenseAdapt: Data-based, Actionable Insight

SenseAdapt can be installed on an on-premise Web Sever (even the TFS box if you wish) or it will be available as a VSTS Extension (see below).  We use the TFS API and so all the data is live and it does not rely on a the data warehouse.  The UI is web based and can be accessed using any device.  All the charts are interactive so you can see much of the data that underpins the chart.

The tool will help teams improve their Agile process but for me, the outstanding feature is the ability to predict project completion dates.

If you have a fixed delivery date, understanding where your project is predicted to finish with the current backlog and completion rate will allow you to make difficult decisions about the scope of the project.  You might not like what you see but it’s better to know ahead of time if there is likely to be a problem with delivery.

Contact me or Bazil if you’re interested in understanding how SenseAdapt can help your team improve.

VSTS Extensions

The TFS Marketplace was in introduced in November 2015 and I expect to see this expand massively over the next 6 months with both free and commercial plug-ins.  We have SenseAdapt almost ready to go (give me a shout if you’re interested in trying it on your account).  At the minute, it is VSTS only but I expect to see extensions on premise in the next few updates. 

VSTS Marketplace

3rd Party BI Tools

We  recently worked with a client who made extensive use of Tableau (and others who use QlikView and Cognos) which seems to be the gold standard in BI tools.

While things started well, the major issue was users understanding the TFS data warehouse.  It could be incredibly difficult to get the data you wanted which kind of defeats the purpose of having this beautiful BI tool. 

So, while you can do it and get good results, it is not an easy option and unless you already have the tools, it can be very expensive.

REST API on Premise

Just a note that TFS 2015 brought the REST API to on-premise TFS. 

“The REST APIs enable a lightweight way to work with Team Foundation Server from virtually any device, platform, or technology stack. You can create and query work items, queue a build, get recent team room messages, access source code, and accomplish almost any team or code management task.”

In fact the TFS API is now available via NuGet in 4 different packages.  Check out Buck Hodge’s blog post on the subject:

NuGet packages for TFS and Visual Studio Online .NET client object model

• Microsoft.TeamFoundationServer.ExtendedClient This package contains the traditional TFS/VSO client object model that uses the SOAP API. You will need it to get full access to the work item tracking object model, version control, test management, and more. If you’ve used the TFS/VSO client object model in applications before, this is the one you’ve used. Because not every API is available in TFS 2015 or VSO currently as a REST API, there are going to be cases where you must use this package. Similarly, there are new features that have been built with only REST APIs for which you will need the Client package.

• Microsoft.TeamFoundationServer.Client Here you’ll find convenient .NET wrappers for our REST APIs for build, team rooms, version control, test case management, and work item tracking. You can of course call the REST APIs directly without using this library. You will encounter cases where an API is not available in this package and have to also use the ExtendedClient package.

• Microsoft.VisualStudio.Services.Client If you need to access account, profile, identity, security, or other core platform services, you’ll need this package.

• Microsoft.VisualStudio.Services.InteractiveClient This package provides the library necessary to show a user an interactive prompt for credentials to sign in. If you are using basic authentication, personal access tokens, or OAuth, you won’t need this.

Richard

Tweet.

A common scenario I encounter is when a client has some existing assets in TFS or VSTS and they want a clean slate, to consolidate backlogs or to use a new Process Template.  More recently it is because they want to move their backlog to Visual Studio Team Services (formerly Visual Studio Online)

If you want to migrate source code history then you’re looking at something like the TFS Integration Tools for on premise TFS or the OpsHub Visual Studio Online Migration Utility for VSTS.  They will also migrate Work Items and their links. 

There are other open source tools for migrating Work Items out there such as Total TFS Migration but they are not always maintained and there are some commercial tools which can be expensive.

If the situation is right then I often find Excel is the simplest mechanism to copy Work Items from one Team Project to another.  I have also used the same technique to successfully move items from Jira, Trello and other systems.

However, using an Excel input list is not a perfect solution on its own.  Off the top of my head, you’re going to have to consider:

• New Work Item IDs
• Work Item Links
• Creation State
• Hyperlinks
• Attachments
• Lost Work Item history
• Discussions
• Test Steps
• Created Date
• Closed Date
• Area Paths
• Iteration Paths

We’ll look at some of those later in this post but let’s get some Work Items copied to begin with.

The simplest scenario is if both source and target projects use the same Process Template so that all your fields match (eg. copy the value of Microsoft.VSTS.Scheduling.Effort from source to target). 

If the templates don’t match then you’ll have to do some planning to understand how the fields match  (for example Microsoft.VSTS.Scheduling.StoryPoints to Microsoft.VSTS.Scheduling.Effort) or fields that might be missing (eg. Microsoft.VSTS.Scheduling.CompletedWork or Microsoft.VSTS.Scheduling.OriginalEstimate).  If the fields are missing then you could consider adding them to the target process template, storing the information somewhere else or simply leave it behind.

Create a Query

First up we need a query to return all the items you want to copy. 

Ask yourself, are you going to copy closed, done or removed Work Items.  Often clients do but don’t realise that it’s not going to give them the historical view that they expect as the Closed Dates will be different.  I’d encourage you to leave them behind and retain the old project for a while.

If you have any links between Work Items that you wish to preserve then it needs to be a Tree Query.  We could hook up links with PowerShell later on (mental note to blog about that later) but I like to use Excel if possible.

A query like this is going to bring back everything in a hierarchical structure regardless of the State or Work Item type. 

Something like this will omit Done or Removed Work Items but just be careful that you don’t lose orphaned items or perhaps an odd situation where you have active child items whose parents have been moved to Done.

You may need to play around with the query to get what you want. 

Importantly, click on Column Options and choose which fields you wish to import.

Then choose your sort order.  Backlog Priority followed by ID is probably a good shout.

Copy your Query to the target project

We have our query to export the data we want but now we need to add that to the target project so that we can import the data.

Open the source project in Visual Studio/Team Explorer and navigate to your query.

Click on Edit Query

And Select File-> Save As…

We want to save our query to a file (*.wiq), somewhere we we can find it.  Open the file in a text editor of your choice (I like Notepad++) or you can do it in Visual Studio as long as you select the Open With drop down.

Edit the Server URL and the Team Project Name in the wiq file and save it again. 

Now open Visual Studio and connect to your target team Project. 

Select File->Open->File…

Navigate to your *.wiq query file.

Select Save As…

and select your new project as the target server. 

The query should now be available in the target Team Project.

Open your Query in Excel

Open Excel, navigate to the Team tab, hit New List and select your query in the source Team Project.

Open another instance of Excel (hold down the Shift key when you click the shortcut) and open the query in the target Team Project.

In the source project we can see the Work Items we are going to migrate.  Note that the Parent/Child hierarchy is represented by multiple Title columns.

This hierarchy is missing on the target query so we need to add as many levels as we need.

On the Team ribbon, select Add Tree Level, in my case I needed three more levels to my hierarchy to represent Epic-Feature-PBI/Bug-Task.

Create new columns

Now we need to create some space where we can store related information that we will not publish to the new team project.  You could add custom fields in TFS to store this information but I like to keep the template as clean as possible.

Right click on the column in Excel and Insert columns where you need them.

Repeat the process to store things like the Old Id, the Old State (all new items will be created as New/To Do etc), Old Area Path and so on.

I like to colour these columns (it’s always orange for some reason) so that it is clear that they will not be published back to TFS and only live on this spreadsheet.

Clicking Choose Columns in the Team ribbon allows you to re-order the columns including the non-TFS ones.  This will make it easier to copy and past in the next section.

Copy source Project data

Begin the process of copying the data from the source sheet to the target sheet.  When we hit Publish on the Team menu, new Work Item IDs will be generated for us in the new project and the Parent-Child links will be created.

Edit the copied data

Now we need to alter some things like the Work Item State.  The TFS Process Template will not allow us to create Work Items in a non-starting state so our items will be New or To Do for example.  We need to filter on the Old States we want to edit.  Click the drop down on the column header and select the first state to edit.

Now we see only the items we need to change.  Change and then fill down the value in the State column (In Progress in the example below) and publish back to TFS.  Repeat the process for all the states returned in your query

You will also need to come back and do this for things like Area Path and Iteration Path if you haven’t already created them in your new project.  Just save this spreadsheet and you can refresh and publish from/to TFS anytime.

Conclusion

So now we have our basic data in our new Team Project along with the Work Item hierarchy.  For many this will be enough but for others, there will be hyperlinks to SharePoint documents, attachments and more that need to be migrated.

Colin Dembovsky has a great blog post - Bulk Migrate Work Item Comments, Links and Attachments – which provides a PowerShell script which is really helpful. 

I want to talk about some of the other issues such as Test Steps and Area Paths in another article but for now it’s time for Christmas.

Cheers,

Richard

Tweet.

Writing queries in TFS is really simple, right?

• Flat query showing high severity Bugs with low effort estimation in the modelling area?

“No problem.”

• Direct Link query showing only Backlog Items that do not have an associated Test Case in a Ready state? 

“Here you go…”

• Tree query showing a hierarchy of Epics with a high business value with Features, PBIs and Bugs that are in Release 2? 

“Errr, let me think about that!”

That’s why with TFS 2013 I always found it easiest to open up my Backlog, change the view to show a tree, for example Features to Tasks…

Then I could hit that little Create Query button…

name my query…

And Bob’s your uncle, I have a starting point Tree Query generated for me that I can mess around with to get what I need.

I did this as normal on Visual Studio Team Services (formerly Visual Studio Online or VSO) the other day and to my horror, my generated query didn’t show the hierarchy I was looking at in my backlog view meaning that I actually had to think about it for the first time in a while!

In VSTS/TFS 2015, the backlog view has changed a little and now the Parent/Child view is either On or Off.

The create query button is still there and it generates a tree query but it only shows Product Backlog Items or Bugs with any nested items the have (which I don’t encourage you to do, keep that backlog flat as you can’t reprioritise nested items independently).

So I had to generate my own starting query.

In this instance I just wanted EVERYTHING, but in a hierarchical view so I started here:

What this tree query is doing is to match the top level clause first (Any work item) and then match any child items which meet the criteria (Any work item) which in this case showed Epics – Tasks, ignoring any state or other criteria such as Area or Iteration Paths.

So what changed?

If I look at the generated query from TFS 2013 we can see what it is doing (and understand why TFS 2015 doesn’t work that way anymore)

Remember this query was looking at Features and then any linked Features, Bugs, PBIs or Tasks.  Any PBI not rolling up to a Feature would not appear in the results.  Also, importantly the query is sorted by Backlog Priority followed by ID.

If we look at an alternative view which is Backlog Items to Features, what it will show is my entire backlog of PBIs/Bugs and any parent Features.  Any Features that do not have child Requirements will not show up in the results.

The difference here is that we are comparing the linked Work Items first (any PBIs/Bugs meeting the criteria) and then looking for any parent items that they have that are either more requirements or Features.

How do I reproduce it in VSTS?

In TFS 2015/VSTS the generated query I initially expected is probably closer to my Any Item linked to Any Item query, just with a few restrictions.  For instance my top level query might look something like this if I want to see Features, Bugs, PBIs and Tasks and any Children they have.

Think carefully about what you want to see at the top level.  Maybe you only want to see Features (or Epics or whatever) and the tree below them – and then have a separate query for any orphaned PBIs, Bug or Tasks.  There are lots of edge cases such as a PBI in an active state but its parent is closed so make sure you’re not losing things.

Good queries are really only useful against a well organized backlog.  If you need help to organise your backlog and really understand how your project is doing then get in touch with RippleRock.

Cheers,

Richard

Tweet.

Are you adopting TFS 2013, 2015 or even Visual Studio Online?

Do you need to train a lot of staff members with different roles?  eg. developers, testers, scrum masters, product owners, stakeholders.

Is it difficult to allow team members to be out of the office for 2, 3 or 5 consecutive days at a training course.

RippleRock can customise a flexible TFS training course to suit you and if required combine it with consultancy, coaching and certification (eg. Certified Scrum Master, Certified Product Owner).

We typically deliver training onsite and configure the agenda and schedule to suit your requirements meaning it is a much more cost effective of training the right people on the right topics.

RippleRock are a Microsoft Gold Application Lifecycle Management Partner and our trainers are ALM Consultants with Microsoft Certified Solution Developer: Application Lifecycle Management

    

As an example of potential topics:

Introduction and overview

• What is Application Lifecycle Management?
• What is Team Foundation Server?
• Visual Studio Tools line up
• Process Templates & Work Items
• Choosing a Process Template
• Understanding Team Projects & Project Collections
• Client Access and Tools
• Scrum Process re-cap
• The Visual Studio Scrum Process Template

TFS Architecture & administration

• Planning a TFS  Deployment
• Installing Team Foundation Server
• Migration/Upgrade considerations
• TFS Administration Console
• Setting up SMTP & Email Alerts
• Monitoring Server Health
• Backup and Restore
• Administration via Command Line and PowerShell
• TFS Security and Permissions
• Rebuilding the TFS data warehouse and adding reports

Agile Planning

• Agile Portfolio Management
• Gathering requirements, estimation and prioritisation
• PowerPoint Storyboarding
• Backlog maintenance
• Forecasting and release planning
• Sprint Planning
• Capacity Planning
• Agile Boards (Task & Kanban)
• Working with Teams
• Team Rooms
• Work Item Queries
• Work Item Charting
• Team Web Access Reporting
• Stakeholder feedback

Agile Development

Visual Studio & Team Explorer   

• My Work
• Suspend/Resume
• Code Reviews
• Code Metrics
• Code Clone Analysis
• Unit Testing and code coverage

Team Explorer Everywhere

• Cross platform version control
• Automating builds with Nant or Maven

Version Control Basics

• Choosing a Version Control System
• Team Foundation Version Control Vs. Git
• Version Control Settings
• Check-in policies
• Check-in notes
• File types
• Workspaces
• Server Vs. Local
• Check-in/out
• Changesets
• Shelving

Advanced Version Control

• Branching & Merging
• Branching Strategies
• Branch Visualisation
• Dealing with Conflicts
• Viewing version control history
• Rolling back code changes
• Securing version control
• Using custom difference and merge tools
• Command line version control
• Git fundamentals

Automated build introduction

• Team Foundation Server Build architecture
• Creating a build definition
• Build triggers
• Continuous Integration
• Scheduled Builds
• Gated Check-ins
• Editing Build Parameters
• Including unit tests in your build process
• Queuing a build
• Private Builds

Customising automated builds

• Process customisation
• Using 3^rd Party components
• Including PowerShell scripts
• Versioning Assemblies
• Cloning a build
• Comparing Build Definitions with tfpt.exe

Agile Testing

• Overview of test tools in Visual Studio
• Microsoft Test Manager
• Test Plans/Suites/Cases
• Microsoft Test Runner
• Exploratory Testing
• Web Based Test Manager
• Automated Tests (CodedUI) inc. cross browser
• Microsoft Lab Management
• Web Performance Test and Load Test

Microsoft Release Manager

• Release Manager Overview and architecture
• Licensing Release Manager
• Integrating Release Manager with your build process
• Creating a release
• Using PowerShell Desired State Configuration

Team Foundation Server Reporting

• How data flows through TFS
• Out-of-the-box TFS reports
• Team Web Access Reports
• Work Item Charting
• Work Item Reporting
• Custom Excel Reporting
• Custom SSRS Report

Customising Team Foundation Server

• Understanding Process templates
• Editing Process Templates
• Process Template Editor
• Witadmin.exe
• Customising and creating Work Items
• Working with Global Lists
• Tailoring Agile Portfolio functionality
• Editing the Team process
• 3^rd Party Tools and controls
• Using the TFS API
• PowerShell with TFS

Contact us to discuss your requirements and build a personalised training plan.

Richard

Tweet.

There are some great features coming in Team Foundation Server 2015 and they’re already largely available in Visual Studio online. 

My personal favourite is the enhanced Agile tooling in Team Web Access and in particular the improved support for Kanban (swimlanes, explicit policies etc.).  However, I love the new simplified build system and its tight integration with test and release.  Release Management is also ready for the big time with cross platform support and a new web client.

TFS 2015 is due for release on 20th July and there are a lot of reasons you should think about upgrading.  In addition, editable process templates are coming to Visual Studio Online so you might also be thinking about a move to the cloud.

However, do you find yourself running TFS 2010 or TFS 2012 (even 2005 or 2008 if you’re really unlucky) and now you’re one more version behind which can be a problem for supportability?

Maybe you have a heavily customised process template or you’re using Scrum For Team System and you want to change it as part of the upgrade?

If you need some help and advice on the options open to you then drop us a line. 

We can help assess your situation and leave it to you from there or take care of the whole upgrade process and follow up with training and coaching.  We’ll tailor any engagement to suit your requirements.

Richard

Tweet.

I use a local TFS 2013 HyperV image for most of my demos and experimentation and so reasonably frequently I break it and just rollback to a snapshot rather than spending time trying to repair it.

This week I wanted to keep some work on there but I’d also broken a collection (on purpose) so my plan was to delete the collection and re-create it (I could have just created a new one but I liked the succinctly named TrainingCollection!)  

These are the steps I followed

• Launched the TFS Admin tool and detached TrainingCollection
• Opened SQL Server Management Studio and deleted the Tfs_TrainingCollection database
• Created a new collection called TrainingCollection using TFS Admin

I then created a Team Project and it failed at the Reporting Services stage because a TrainingCollection folder already existed.  Oops. 

• Navigated to http://<tfsserver/reports
• Clicked on TfsReports and deleted TrainingCollection

I tried creating the Team project and it failed again, so I had a look in the build creation log

---begin Exception entry---

Module: Engine

Event Description: TF30162: Task "WITs" from Group "WorkItemTracking" failed

Exception Type: Microsoft.TeamFoundation.Client.PcwException

Exception Message: TF24016: Cannot find team project 'CustomerTraining'.

--- end Exception entry ---

I realised that I had confused Visual Studio with all my messing around so the way forward was to delete the local TFS Cache so that Visual Studio could forget about the old collection and get to know the new one.

• Closed Visual Studio
• Navigated to my cache folder and deleted the contents.  

For my installation it was here:

C:\Users\<MyUser>\AppData\Local\Microsoft\Team Foundation\5.0\Cache

Obviously, it’ll be slightly different if you’re targeting another version of TFS or if you have installed VS on another drive.

I re-opened Visual Studio and created my Team Project with no further issues. 

I don’t suppose this is something you’re likely to run into in a production TFS environment but  you never know!

Cheers,

Richard

Tweet.

Final post for tonight and then I might have a little nap on the train.

On another recently created project utilising the custom Team field to scope Work Items to TFS teams rather than the Area Path I encountered another minor issue.

I created a Test Plan using Microsoft Test Manager and then clicked the Open in Web Access button

I received the following error in Team Web Access:

The test plan with id XX does not exist or it’s area path is not owned by the default project team.  Include it in the default project team’s owned areas and try again.

The error is a little cryptic as I hadn’t even started adding area paths, much less locking them down.  It was in fact, another symptom of using the custom Team field,

I hadn’t implemented the template customisation and so the Work Item definition was missing the following on the Team field:

<DEFAULT from="value" value="Unassigned" />

TFS 2013 Update 3 requires that you add your Team field to the new Test Plan Work item.  You don’t have to expose the field on the UI but it must have a default value or you will hit issues in Team Web Access when accessing those Test Plans that do not have the field populated.

Easy to resolve –

• Run a query into Excel that brings back all the Test Plan Work Items
• Add the Team field to the list of columns
• Set the Team value for the Test Plans to Unassigned (or whatever you use)
• Use witadmin.exe or the Process Template Editor to update your Work Item definitions to have a default Team value

Time for a snooze.

Cheers,

Richard

Tweet.

If you made it through my last post, you’ll know that I recently updated a TFS 2013 RTM install to Update 2 to resolve a bug in TFS. 

During the update we have to re-apply service accounts and confirm servers etc.  At one stage we were prompted for the SQL Reporting Services Server and the reporting service password.  The issue we needed to resolve was holding up work and unfortunately we couldn’t locate the SSRS password that we needed.  I asked which was more important – getting the team working again or running reports.  The answer was obviously the former and so we took the decision to skip Reporting Services safe in the knowledge that we could deal with it later as a non-critical issue.

The next day the TFS admin had all the details he needed to re-enable reporting and got stuck into the TFS Admin Console.

MSDN: TFS 2013 - Add a report server

However, he added the server details for the data warehouse, clicked Test Connection and received the following error:

The database you specified cannot be used. The database exists, but its schema is not valid for use with Team Foundation Server.

The obvious reason for the error was that the TFS update had changed the data warehouse schema and now our existing warehouse database was out of date.

I wondered if we could update the warehouse database individually but if you understand the reporting architecture in TFS (read my other blog post if you don’t) then you’ll know that we could just blow away the existing warehouse and re-create it.  So that’s what we did, we now have the imaginatively named Tfs_Warehouse2 and Tfs_Analysis2 databases.  We hooked up reporting again, rebuilt the warehouse using the Admin Console and Bob’s your uncle.

We hit a minor snag later that the analysis database didn’t seem to have reflected the changes so we called the ProcessWarehouse web service followed by the ProcessAnalysisDatabase web service, checked everything completed successfully with the GetProcessingStatus web service (details here) and everything was working again in time for tea and biscuits!

I later found an old TFS 2010 related blog post that confirmed what had happened and that our resolution was correct.  Hurrah!

60 minutes left, let’s go for the hat-trick.

Cheers,

Richard

Tweet.

My blog posts are like really terrible buses, you wait 8 weeks for one and now I’ve finally got a little time (3 hours on a train home from the midlands) so 2 or 3 are going to arrive at once.

A process template customisation I do regularly for customers is adding a Team field to scope Work Items to teams in TFS 2012 and TFS 2013.  Microsoft have detailed the process here so it’s not a particularly controversial thing to do and I believe it provides a better experience when you have multiple teams working against a central backlog.

I blogged recently about how to use PowerShell to automate the process of adding a team field.

I performed the customisation for three companies in the last couple of months and bumped into a some issues with Team Web Access in particular.  Even though adding a Team field is very much a valid thing to do, there can be a couple of wrinkles to iron out.

The first issue came as a bit of a shock.  The background to the problem is this - a customer has been happily using TFS 2013 on a pathfinder project with a single team.  Things have gone well and so they wish to roll out the tool to about 5 other teams.

We worked together to create a slightly customised process template which added some further hierarchy to the standard Visual Studio Scrum 2013 process template.  Without thinking, I started work using a Visual Studio Scrum 2013.3 template and when I tried to upload it to the customer’s server we had some errors as Update 3 added all sorts of test related artefacts that weren’t available with the customer’s TFS 2013 RTM version. 

That wasn’t the issue, however and I quickly made the changes to an RTM version of the VS Scrum template and uploaded it successfully.  We then created a central project to house the company’s portfolio backlog (I must add that to my things to blog about!).

We were to resume working on the new project a few days later but before then I received a frantic email from the pathfinder team saying that whenever they tried to edit an existing Work Item using the Team Web Access client they received the following error:

Team Foundation Error

The server operation failed, (Value cannot be null. Parameter name:key).

I quizzed the team on whether they had changed anything on the pathfinder project as templates are like cookie cutters, once you cut out the cookie, it doesn’t matter what you do to the cutter, it doesn’t effect the cookies you produced previously.

I went onsite and pulled down the processconfiguration.xml, categories.xml and the Work Item definitions for the pathfinder project.  Nothing had changed!

Some experimentation confirmed that uploading a new process template configured to use a custom team field had broken Team Web Access for an existing project.  Clearly that shouldn’t happen and although I we could have battled through and fixed it we took the decision to update the TFS 2013 RTM server (without knowing for sure that Update 2/3/4 had fixed the problem).  We took the decision to go with Update 2 as we weren’t quite ready for the new test artefacts and I’d recently been badly burned by an issue with Update 3 massively inflating the collection database (another thing to add to my list of blog posts to write).

The upgrade to Update 2 only caused us a short period of downtime and it was something we were planning on doing soon anyway.  The problem was resolved immediately so it was obviously a known bug which was addressed in the first update for TFS 2013 (there was no update 1 for TFS 2013).

A similar but not identical issue is discussed in the MSDN forum post here.

There are still 90 minutes left of my train journey so I may as well write up the next instalment.

Cheers,

Richard

Tweet.

One of the Team Foundation Server Process Template customisations I do frequently is changing a project to make use of a custom team field rather than the default Area Path to scope Work Items to teams.

If you haven’t made use of teams in TFS 2012 or TFS 2013 yet then check out these links

Multiple Teams with Microsoft Team Foundation Server 2012 & Visual Studio Scrum

TFS 2013: Collaborate using team resources

If you don’t know what is involved in customisation then you need to check out this page.

Customize a team project to support team fields

Now, you can, as with any process update, make changes offline and any subsequent projects created using that template will see the changes.  Or, you can update an existing in-flight project with template changes. 

It’s not particularly difficult to change your template to make use of a team field but it is time consuming so I wrote a PowerShell script to do it both to save me time and to play around with using PowerShell to do XML transformations.

My script downloads the required files from a Visual Studio Scrum 2013 project, backs them up, performs an XML Transform on them and uploads them to your project.  If you are using a different process template then you will need to edit the XML transforms and the selected Work Item definitions but the technique is the same.

You can call the script Add-Teams.ps1 from the command line and provide 3 parameters

Add-Teams.ps1 “http://TFS2013:8080/tfs/DefaultCollection” MyProject “C:\TemplateUpdates”

Defaults are specified at the beginning of the script so if you plan to run it using PowerShell ISE for example, then simply change the values.

        #TFS Collection Url
        [string]$CollectionUrl =          "http://TFS2013:8080/tfs/DefaultCollection",
        #Name of the TFS Team Project to transform
        [string]$TeamProjectName =        "MyProject",
        #Working folder to store the XML transforms, downloaded files and backup
        [string]$WorkingFolder =          "C:\TemplateUpdates" 

There is some (hopefully) useful logging throughout the script which uses write-verbose, so if you want to turn it on then  give the script a –verbose flag when you run it.

The script

First we create a function called Transform-XML that takes an XML file and a transform file to apply.  We’ll use it to alter the Work Item definitions, the Process Configuration and add a Global List.

---

function Transform-XML 
{
     Param (
            [Parameter(Mandatory=$True)][ValidateNotNull()]$Wit,
            [Parameter(Mandatory=$True)][ValidateNotNull()]$Tfm
        )

---

Next, the necessary Work Item definitions (Product Backlog Item, Task, Bug, Feature & Test Plan) are downloaded from your project using WitAdmin.exe (usually found in C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Witadmin.exe)

---

# Export Work Item definitions 
write-Verbose "Exporting PBI definition"
& $WitAdmin exportwitd /collection:$CollectionUrl /p:$TeamProjectName /f:$PBIDefinition /n:"Product Backlog Item"
write-Verbose "Exporting Task definition"
& $WitAdmin exportwitd /collection:$CollectionUrl /p:$TeamProjectName /f:$TaskDefinition /n:"Task"
write-Verbose "Exporting Bug definition"
& $WitAdmin exportwitd /collection:$CollectionUrl /p:$TeamProjectName /f:$BugDefinition /n:"Bug"
write-Verbose "Exporting Feature definition"
& $WitAdmin exportwitd /collection:$CollectionUrl /p:$TeamProjectName /f:$FeatureDefinition /n:"Feature"
# Required if using TFS 2013 Update 3 or later 
write-Verbose "Exporting Test Plan definition"
& $WitAdmin exportwitd /collection:$CollectionUrl /p:$TeamProjectName /f:$TestPlanDefinition /n:"Test Plan"

# Export Process Configuration
write-Verbose "Exporting Process Configuration"
& $WitAdmin exportprocessconfig /collection:$CollectionUrl /p:$TeamProjectName /f:$PCDefinition 

# Export Global List 
write-Verbose "Exporting Global List"
& $witadmin exportgloballist /collection:$CollectionUrl /f:$GLDefinition 

The we copy all the downloaded files into a backup directory.  If your Team project was called MyProject then all the files would be downloaded to C:\TemplateUpdates\MyProject and a backup would be copied to C:\TemplateUpdates\MyProjectBackup\110914.161115 where the date is 11\09\2014 and the time is 16:11 and 15 seconds.  This way, if anything goes wrong, you have your original files to upload.

---

# Create backup of process files 
write-Verbose "Backing up process files"
$timestamp = Get-Date -UFormat "%d%m%y.%T" | foreach {$_ -replace ":", ""}

Copy-Item -Path $ProjectFolder -Container -Destination "$($ProjectFolder)Backup\$timestamp" -Recurse -force

After that we call the Transform-XML function against the required files

try
{
    #Try transforming the required files and exit the script if there are any problems
    
    # Add Global List 
    write-Verbose "Adding Team Global List"
    Transform-XML $GLDefinition $GLTransform

    # Update Work Item Definitions to add new Team field and include on form 
    write-Verbose "Updating PBI Definition"
    Transform-XML $PBIDefinition $WITTransform
    write-Verbose "Updating Task Definition"
    Transform-XML $TaskDefinition $WITTransform
    write-Verbose "Updating Bug Definition"
    Transform-XML $BugDefinition $WITTransform
    write-Verbose "Updating Feature Definition"
    Transform-XML $FeatureDefinition $WITTransform
    # Required if using TFS 2013 Update 3 or later
    write-Verbose "Updating Test Plan Definition"
    Transform-XML $TestPlanDefinition $WITTransform

    # Update Process Template to make use of new Team field instead of AreaPath 
    write-Verbose "Updating Process Configuration"
    Transform-XML $PCDefinition $PCTransform
}

The XML transform for a Work Item looks like this for example

---

<?xml version="1.0"?>
<witd:WITD xmlns:witd="http://schemas.microsoft.com/VisualStudio/2008/workitemtracking/typedef" xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
  <WORKITEMTYPE name="Bug">
    <FIELDS>
    <!-- If the team field has already been added then remove it first-->
    <FIELD name="Team" xdt:Transform="Remove"  xdt:Locator="Match(name)" />
      <FIELD name="Team" refname="RippleRock.Scrum.Team" type="String" reportable="dimension" xdt:Transform="Insert">
        <HELPTEXT>Name of the team that will do the work.</HELPTEXT>
        <ALLOWEXISTINGVALUE />
        <ALLOWEDVALUES >
          <GLOBALLIST name="Teams" />
        </ALLOWEDVALUES >
        <DEFAULT from="value" value="Unassigned" />
      </FIELD>
    </FIELDS>
    <FORM>
      <Layout>
      <Group>
      <Column>
        <Group Label="Status">
          <Column PercentWidth="100">
          <!-- If the team field has already been added then remove it first-->
            <Control FieldName="RippleRock.Scrum.Team" xdt:Transform="Remove"  xdt:Locator="Match(FieldName)" />
            <Control FieldName="RippleRock.Scrum.Team" Type="FieldControl" Label="Team" LabelPosition="Left" EmptyText="&lt;None&gt;" xdt:Transform="Insert" />
          </Column>
        </Group>
        </Column>
        </Group>
      </Layout>
    </FORM>
  </WORKITEMTYPE>
</witd:WITD>

---

This adds a field called RippleRock.Scrum.Team to the Work Item and then adds a Field Control at the end of the Status group on the Work Item form.  I added an additional Remove transform for both elements so that if you run the script against an item that already has a Team field then it will be replaced.

Finally, we import all our definitions and we should be good to go.

---

# Import Global List 
write-Verbose "Importing Global List"
& $witadmin importgloballist /collection:$CollectionUrl /f:"$ProjectFolder\GlobalList.xml"

# Import Work Item definitions 
write-Verbose "Importing PBI definition"
& $WitAdmin importwitd /collection:$CollectionUrl /p:$TeamProjectName /f:"$ProjectFolder\WorkItem Tracking\TypeDefinitions\ProductBacklogItem.xml" 
write-Verbose "Importing Task definition"
& $WitAdmin importwitd /collection:$CollectionUrl /p:$TeamProjectName /f:"$ProjectFolder\WorkItem Tracking\TypeDefinitions\Task.xml" 
write-Verbose "Importing Bug definition"
& $WitAdmin importwitd /collection:$CollectionUrl /p:$TeamProjectName /f:"$ProjectFolder\WorkItem Tracking\TypeDefinitions\Bug.xml" 
write-Verbose "Importing Feature definition"
& $WitAdmin importwitd /collection:$CollectionUrl /p:$TeamProjectName /f:"$ProjectFolder\WorkItem Tracking\TypeDefinitions\Feature.xml" 
# Required if using TFS 2013 Update 3 or later
write-Verbose "Importing Test Plan definition"
& $WitAdmin importwitd /collection:$CollectionUrl /p:$TeamProjectName /f:"$ProjectFolder\WorkItem Tracking\TypeDefinitions\TestPlan.xml"

# Import Process Configuration 
write-Verbose "Importing Process Configuration"
& $WitAdmin importprocessconfig /collection:$CollectionUrl /p:$TeamProjectName /f:"$ProjectFolder\WorkItem Tracking\Process\ProcessConfiguration.xml"

---

Remember to follow the steps in the “Configure Team Settings” section of the Customize a team project to support team fields to alter how Work Items show up on the team backlogs.

Download the script and the transforms here.

I hope it saves you some time.

Cheers,

Richard

Tweet.

I was working with some custom build definitions on a training course last week and one of the delegates had a failing build.  After a quick visual inspection I couldn’t spot what was wrong and so I wanted a way of easily comparing the build with my working build definition.

I knew we could pull the information using the TFS API and the IBuildDefinition Interface but I came across a flag on the TFS Power Tools (tfpt.exe) builddefinition option that I hadn’t realised was there.

tfpt.exe builddefinition /diff “<Project>\<Build Definition 1>” “<Project>\<Build definition 2>” /collection:<collection url>

This gives us a really quick and simple way of comparing the build definitions and it uses the Visual Studio file comparison to show the differences.

For more information on the command, launch a Developer Command prompt and type:

tfpt builddefinition /diff /collection:<collection url> /?

So, for example on my demo setup

• Server = TFS2013
• Collection = RippleRockCollection
• Team Project = BuildDefinitionTest
• Baseline Build = BuildDefinition1
• Build for Comparison = BuildDefinition2

I used:

tfpt builddefinition /diff “BuildDefinitionTest\BuildDefinition1” “BuildDefinitionTest\BuildDefinition2” /collection:http://TFS2013:8080/tfs/RippleRockCollection

You’ll probably find an error saying:

Could not load file or assembly 'Microsoft.TeamFoundation.Build.Workflow, Version=12.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.

You can find Microsoft.TeamFoundation.Build.Workflow.dll in:

C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\PrivateAssemblies

and simply copy the file to:

C:\Program Files (x86)\Microsoft Team Foundation Server 2013 Power Tools

Tfpt.exe should now launch Visual Studio and make the definitions very easy to visually compare side-by-side.

Cheers,

Richard

Tweet.

*Check out the 2015 update to this post

I’ve been answering a lot of questions on Team Foundation Server reporting over the last couple of weeks so I thought I’d put some of that information in a blog post.

Team Foundation Server is an amazing tool for storing information about your development projects and all that information can be retrieved if you know what the options are.

This post is targeted at TFS 2013 but most of it is relevant for TFS 2010 & TFS 2012

How does data flow in TFS?

This is probably the most concise reference of TFS reporting architecture and the various options that I can find: Create, customize, and manage reports for Visual Studio ALM

 

In a nutshell, when you work as normal in TFS - checking in source code, executing builds, running tests and completing work - all that information is being stored in transactional, operational stores within the Team Project Collection database (there is a separate collection database for each Team Project Collection that you create). 

These stores are designed to provide high transaction speed and data integrity in normalised tables meaning data isn’t duplicated.  As a result, it’s not very easy to work with these tables, it’s difficult to know which tables to use to get the data you want and queries can be slow because of all the joins required. 

A number of data adapters then populate a single, relational, data warehouse with the latest data from the stores.  Note that reportable data from all team project collections is published to that warehouse (Tfs_Warehouse) meaning that you can report across multiple collections (obviously, you should be careful who you give permissions to!)

Tfs_Warehouse has a schema that is much easier to understand because it is optimised for queries and reporting instead of transactions.  Additionally, this database can be on a completely different server so queries won’t slow down TFS.  The data warehouse is organised in a star-like schema into fact tables and dimension tables.

Finally, and every 2 hours by default, an Online Analytical Processing (OLAP) database (Tfs_Analysis) is populated to allow easy reporting on that information and this is where most of the standard reports will pull their data from.

You can read more about the architecture used for TFS reporting in Components of the TFS data warehouse

Types of Report

What types of reporting are available in TFS?

• Work Item Queries
• Work Item Reporting
• Out-of-the-box Excel Reports
• Out-of-the-box SSRS Reports
• Team Web Access Charting
• Team Web Access standard reports
• Custom Excel Reports
• Custom SSRS Reports
• SQL Queries
• TFS API
• REST API

Let’s look at them in a little more detail

Work Item Queries

Work Item queries are probably the simplest way of pulling information from TFS.  The data retrieved is always up to date as queries do not rely on the data warehouse. 

There are 3 types of query

• Flat Queries
• Direct Link Queries
• Tree Queries

For more information on Work Item Queries see Query for work items

TFS 2013 Update 2 adds the option to query on Work Item Tags.  If you have not installed TFS 2013 Update 2 or if you have TFS 2012 then  you cannot include tags in your Work Item Queries

Work Item Reporting

Work Item Reporting is the easiest route into custom reporting with TFS.  You can take a Flat List Query and within Excel click the New Report button on the Team menu

or in Visual Studio, click the Create Report in Microsoft Excel

From there you can choose to generate a report for the current state and/or the trend over time

TFS will generate a workbook with a number of reports on separate worksheets and a table of contents.  You are free to customise any of these reports and save the result for later use.

Here is an example of editing an Excel report Custom Burnup chart with TFS and Excel

For more detail, see Creating Reports in Microsoft Excel by Using Work Item Queries

Out-of-the-box Excel Reports

Depending on your chosen process template, and if you have installed SharePoint as part of your TFS deployment then you may have access to a number of standard Excel reports. 

These are included with both MSF templates (Agile & CMMI) but not with the Visual Studio Scrum template.  That said, with a little tweaking they could work with the Scrum template (eg, swap references to Story Points field used by MSF Agile to the Effort field used by Scrum)

They can be accessed through the SharePoint portal or from the Documents tab in Team Explorer.

Here is the Task Progress report as an example.

See Excel Reports for more detail

Out-of-the-box SSRS Reports

Again, depending on your chosen process template (and assuming you have installed Reporting Services) then you will have access to a number of reports automatically when you create your Team Project.  They can be accessed via Web Access, SharePoint and Team Explorer.

View, upload, and organize reports (Reporting Services Reports)

For example, this is the standard Backlog Overview report available with the Scrum process template

For a full list of available reports see Reports (SQL Server Reporting Services)

To view the reports, you must be assigned or belong to a group that has been assigned the Browser role in SQL Server Reporting Services. For more information, see Grant permissions to view or create reports in TFS.

Team Web Access Charting

Charts are new in TFS 2013 and are a bit like Work Item Reports but a lot simpler to create and not as customisable or detailed.  The big benefit is that you can pin these charts to your Home page in Team Web Access (you’ll need TFS 2013 update 2 for that) so they become very visible to your team

My good buddy, Colin, has an excellent write up on reports here although it pre-dates Update 2 so doesn’t mention pinning to the home page or customising colour on the reports (read about that here)

You can view charts with a Standard TFS Client Access License but to create them you must have a Full license which means Visual Studio Premium, Ultimate or Test Professional.  See more about Access Levels for Team Web Access here.

Team Web Access Charting

Team Web Access standard reports

Team Web Access has some useful reports which can be accessed very easily.  The reports are non-customisable but the good thing about them is that they do not rely on the data warehouse cube so they are never out of date.

Velocity

The Velocity diagram shows the number of Story Points (or whatever unit you are using to estimate User Stories/Backlog Items) scheduled and completed across iterations.  This will help with forecasting and release planning for your project.

Cumulative Flow Diagram

The CFD Diagram shows up to 30 weeks of data displaying the number of Work Items and their state over a date range.  If bulges are showing up on your CFD it could be an indicator of a bottleneck in your development process

TFS 2013 Update 2 adds the ability to specify a start date for your CFD which can be very useful if you have a long running project.

Sprint Burndown

Finally, your Sprint Burndown shows the number of hours (assigned to the Tasks) your team has committed to in the sprint.  As work is completed (or added) then the value for Remaining Work on the Y-axis will change and an ideal trend line will show you when you are likely to complete the work if the current work rate continues

TFS 2013 Update 2 finally added the much requested feature to be able to remove weekends from the TWA Burndown chart.  This means the burndown does not show the “steps” of a flat line when no work is completed at the weekend which could be a little demoralising.  Giles covers it in more detail here

Custom Excel Reports

Now you get into some proper custom reporting.  I still feel that the Work Item Query Reports covered earlier in the article are the easiest way to get started with Excel but you can connect to the Analysis Database manually if you like.

Take a look at Create a Report in Microsoft Excel for Visual Studio ALM which describes how to connect Excel to the TFS Analysis database.  Doing it this way means you are creating a report from scratch which is great if you know what you want but there is definitely a steeper learning curve.  Start with reading Perspectives and measure groups provided in the Analysis Services cube for Visual Studio

Custom SSRS Reports

If you have a SQL Server Reporting Services expert on your team then they may prefer to generate a report that way.  The benefit of using SSRS is that the report can be published to Team Foundation Server and it will be available alongside the OOB Reports in the Reports tab of Team Explorer or the Reports website.  It also allows for much more detailed reports than excel and allows you to plot multiple data points on the same report.

You can use:

• Report Builder
• Report Designer

Both report authoring tools can target Tfs_Analysis and Tfs_Warehouse although you may find Report Designer more complex to begin with.  You can view a comparison of the two tools here.

It may be easier to start with an existing report and customise it.  As well as the OOB reports, there are a number of sample reports available such as the Community TFS Report Extensions

Create a Detailed Report using Report Designer

Table reference for the relational warehouse database for Visual Studio ALM

Perspectives and measure groups provided in the Analysis Services cube for Visual Studio

If you create a custom report then you may want to add it to your Process Template so it is automatically available for any new projects created.

Add reports to the process template

SQL Queries

The majority of the reports we’ve discussed so far pull their data from the Analysis Services OLAP cube but although it’s generally more difficult, there is no reason why you can’t grab data directly from the relational data warehouse

The TFS Reporting Guide is generally a good resource for TFS reporting but it has some good examples of running T-SQL queries directly against the TFS Data Warehouse relational database.  For example, check out the Mean Time Between Failure (MTBF) Report which uses the CurrentWorkItemView to retrieve the current state of each work item from the dimension table.  The sample query fetches all Work Items of type Bug that are in a Closed state in the chosen Team Project and calculates the average time difference in hours between them.

You can also query the TFS operational stores but it’s not advised and you should really use the TFS API instead.  For example How to generate a report of active users who log onto the TFS server

TFS API

One of the great things about TFS has always been it’s customisability and there has always been a reasonably well documented API for pushing and pulling information to/from TFS.  For example you can create a TFS Work Item with about 3 lines of C# code.  You can also use the TFS API to create reports in a supported way.  If you have a look at the WCF Service sample for a Build Duration Report in the TFS Reporting Guide Data Warehouse section.

You’ll need to become familiar with Extending Team Foundation too.

REST API

A little while ago an OData Service was released for TFS to allow clients to be created for smartphones, tablets and non-Windows operating systems as long as they supported http requests. 

Team Foundation Server OData API

OData Service for Team Foundation Server v2

TFS Reporting Guide also has a whole section on this

This has now been enhanced as described in A new API for Visual Studio Online.  However, at this stage this API is only available for Visual Studio Online but it will likely be available for on-premise installations in the next version of TFS.

Some Questions

What about Visual Studio Online?

Visual Studio Online (formerly Team Foundation Service Preview) does not use SQL Reporting Services and so there is no data warehouse to connect the standard reports.  You still have the Team Web Access reports and the option to use the TFS API but in addition it is worth checking out the very cool REST API for Visual Studio Online which should make it easier to pull information from TFS Online

What permissions do I need to view and create reports?

Permissions for Excel Reports That Connect To the Analysis Services Cube

What is the permission needed to view and create a Excel report for TFS Analysis Services cube?

If you prefer to use Report Builder to edit or add Reporting Services Reports then it is slightly different, you’d need to have your account added to the Content Manager role for Reporting Services.

Grant permissions to view or create SSRS reports in TFS

I’m using Microsoft Test Manager with the Visual Studio Scrum template and the OOB reports are lacking.  What can I do?

There are some standard reports for testing such as the Test Plan Progress Report and the Test Case Readiness Report.  If that is not enough then it is worth checking out the (now slightly old but still relevant) blog posts here:

Test Case Management (TCM) Reporting - Frequently Asked Questions - Part 1

Test Case Management (TCM) Reporting - Frequently Asked Questions - Part 2

which gives a lot more information on custom excel reporting.  

Are there metrics associated with Builds and Continuous Integration and can we access historic build success ratios and failures etc.?

There is a full build log with diagnostics available for all builds (you can set a retention policy on them) and also the information is published to the data warehouse.  There are OOB reports (Build Summary, Build Success over Time) and options for custom reporting as discussed above.  See Analyze and report on build details and build coverage using the Build perspective for more detail.

I want to my reports to update more frequently

You can change the process control setting

Managing the data warehouse and analysis services cube

Change a Process Control Setting for the Data Warehouse or Analysis Services Cube

My TFS data warehouse is broken

Check your account permissions, review log files and rebuild the warehouse.

Manually process the data warehouse and analysis services cube for Team Foundation Server

TFS Admin Console - Rebuild the Data Warehouse and Analysis Services Cube

TFSConfig Comman Line - RebuildWarehouse Command

I don’t have Reporting Services or SharePoint installed

No problem, get them installed, start generating your reports or sharing your Excel workbooks with your colleagues.

Install Reporting services

Install SharePoint

Upload reports to a team project

Summary

Hopefully someone will find this post useful and I know I am likely to redistribute a lot of the information within it in the future.  You can see that there are many options for pulling data and generating reports from Team Foundation Server and you just need to figure out which best suits your needs.

Thanks,

Richard

Tweet.

I was doing a little custom reporting using Report Builder today and when I came to preview my report I received an error message.

The permissions granted to user 'TFS2013\user' are insufficient for performing this operation.

System.Web.Services.Protocols.SoapException: The permissions granted to user 'TFS2013\user' are insufficient for performing this operation. ---> Microsoft.ReportingServices.Diagnostics.Utilities.AccessDeniedException: The permissions granted to user 'TFS2013\user' are insufficient for performing this operation.
   at Microsoft.ReportingServices.Library.ReportingService2010Impl.CreateReportEditSession(String Report, String Parent, Byte[] Definition, String& EditSessionID, Warning[]& Warnings)
   at Microsoft.ReportingServices.WebServer.ReportingService2010.CreateReportEditSession(String Report, String Parent, Byte[] Definition, String& EditSessionID, Warning[]& Warnings)
----------------------------
The permissions granted to user 'TFS2013\user' are insufficient for performing this operation.

That seemed odd as I hadn’t messed around with security settings and it was working recently.  Still, just to be on the safe side, I reviewed the article:

Grant permissions to view or create reports in TFS

Everything looked fine so I spent some time playing around with security settings and doing a little web searching.  I found an old stackoverflow post which talked about deleting the encryption key and lots of other things that seemed unlikely.

So I grabbed a cup of coffee and went outside for some fresh air and the answer hit me!

Simple as that, and kind of obvious - Run Report Builder as administrator.

Doh!  Hopefully this blog post will save somebody a little time and at least it has broken my blogging drought. 

Cheers,

Richard

Tweet.

I was delivering a training course on TFS 2012 this week and one of the delegates came across some unexpected behaviour that I hadn’t noticed before.

We had added the Work Items Check-in Policy to version control earlier in the day and were looking at automated builds.

With TFS you can perform a private build (or buddy build) to ensure that your proposed changes integrate successfully before you check them in.  You can also select the option to Check in changes after successful build which turns it into a kind of voluntary Gated Check-in.

This is what my delegate did and as his build completed successfully, his shelveset was committed to version control.  Somebody then pointed out that as the changes were not associated with a Work Item there should have been a policy warning like the one below.

I can’t say I have ever noticed this behaviour before and it took me a minute to realise the reason behind it.  Check-in policies are evaluated on the client side so for example if you use the Code Analysis Policy then it is your responsibility to run the code analysis rules against your code before check-in.  The server will not evaluate the policy for you, it will simply check to see if you have passed the rules locally.  The check in that happens as part of the private build happens server side so it did not have to pass the client side check-in policies.

You can of course control permissions on builds or ensure that the build also runs a suite of unit tests but it’s one to be aware of.

Richard

Tweet.

Recently I’ve been working with a customer on migrating their existing Team Project to a new Process Template on Team Foundation Server 2012. 

We’ve decided to go with the TFS Integration Tools to move selected Work Items and code across to a new project in a new Team Project Collection.  There were a number of alternative options open to us but this is a large project that has been migrated through the versions of TFS and we’re going to take the opportunity to clean it up and consolidate.

I spent some time mapping their existing Microsoft Solutions Framework for Agile Software Development 6.1 project to Microsoft Visual Studio Scrum 2.2.  I had to take into consideration some custom fields that had been added to the template and also some legacy fields that had been left hanging around after previous migrations.

The customer is not a heavy Visual Studio user, doing most of the development through Eclipse and as a result they are licensed for Visual Studio Professional 2012.  In turn this means that their developers have “Standard” licenses for Team Web Access. 

As part of the work we have been doing to transform their development process we have a need to use both the Product Backlog view and the Kanban board, both of which are only available to users who are “Full” licensed users of TWA. 

The cost of upgrading the developers to Visual Studio Premium was prohibitive as they will not make use of the additional Visual Studio features and the current Microsoft licensing did not give us any other option.

I evaluated both Urban Turtle and the Eylean products and although they both have their plus points and are very economically priced, neither gave us exactly what we were looking for.

Last week at the Build Windows Conference, Microsoft released a Preview of Team Foundation Server 2013 which also has a Go-Live license. 

Ordinarily it would be a pretty gung-ho decision to move your business onto pre-release software but in this case it is worth considering.  The reason is that Microsoft have listened to the feedback and re-jigged the licensing for TWA. 

TFS2013 now provides both the Kanban Board and the Backlog & Sprint Planning Tools to “Standard” users (VS Pro) and rightfully provides some very cool additional features (Portfolio Management, Team Rooms, Test Case Management, Feedback Management) to “Full” users (VS Premium, Ultimate & Test Pro) 

   

TFS 2012 Team Web Access Levels

TFS 2013 Preview Team Web Access Levels

As a result, there is a strong case to move the project to TFS2013 and although the decision on which way to go has not been made yet, I was worried that the TFS Integration Tools mapping that I had written would need a lot of work to update it for the Microsoft Visual Studio Scrum 3.0 Template.

I downloaded the Scrum 2.2 and Scrum 3.0 Templates and performed a Diff on the folders.

There were some minor build changes and some changes to allow for the new Git support but I was particularly interested in the Work Item and Reporting changes. 

Release Burndown report – Order by Iteration Path

Minor change but hopefully a helpful one as I have run into this problem before and presumably this small change to the rdl file will resolve it.

New Category – Feature Category

New Work Item Type - Feature

Both of these have been added to support the new portfolio management functionality in TWA.

And that’s it, so there are no field changes to the core Work Items.  Obviously it is still a Preview version of the template but it shouldn’t change significantly and I can make any tweaks if it does.

If you’re in a similar position in that you would like some of the “Full” features from TWA 2012 but the Premium upgrade does not make financial sense then TFS 2013 could give you what you need.

Cheers,

Richard

Tweet.

Microsoft announced that there will be a new Visual Studio and Team Foundation Server this year along with some other cool updates.  Some links in case you missed them:

Brian Harry: Visual Studio 2013 – A good overview of what’s coming up in the next version

Introducing Cloud-based Load Testing with Team Foundation Service – No need to go looking for tin to run those Load Tests now.

Improving Communication with TFS Team Rooms – Nice feature if your teams are distributed

Video: Application Lifecycle Management with Visual Studio 2013 – Good demo from the two Brians at MS.

TechEd North America 2013 – Check out all the on-demand recordings from TechEd

Testing Improvements in Team Foundation Service Update for TechEd 2013 – Updates to web based test management

Extending Visual Studio Team Explorer 2012 – More extension points for VS.

Team Foundation Task Board Enhancer: version .7.6 released – No feature updates but still one of the very few TWA extensions available.

Microsoft Code Digger – Updated this week so it can run against code not contained in Portable Class Libraries. Thanks for the heads-up Col

Cheers,

Richard

Tweet.

Are you using TFS 2012? 

Are you using or adopting Scrum?

Then get the most out of your tools and your process by attending a 2-day Certified ScrumMaster course with a free one day Scrum with TFS course.

The CSM course is taught by our Certified Scrum Trainer, Colin Bird, and I will be helping out with the optional, free third day which will give you hands on experience with TFS 2012, putting into practice what you have learned in the first 2 days.

The course will be held on Microsoft Campus in Reading on the 21st May – 23rd May

More details available here or you can register here.

If you hurry you can still get the discounted early bird price!

Cheers,

Richard

Tweet.

It’s my pleasure to be part of Microsoft UK Tech Days Online 2013 which is happening next Wednesday (24th April 2013).  It’s shaping up to be an excellent 3-day online event for both developers an IT Professionals with some really interesting areas being covered.  Day One is aimed at the development community with sessions on targeting Windows 8 and Windows Azure alongside Team Foundation Server and Application Lifecycle Management.  The day starts at 09:20 BST 

Introduction to Tech Days Online

Developing in Visual Studio, what's changed in the last 12 months?

Developing for Windows 8 with Visual Studio 2012

Interview with Planky/David Gristwood - Visual Studio Tools with Windows Azure

Being Agile with Team Foundation Server 2012

TFS for everyone – Using TFS in a heterogeneous project (Eclipse, Java and using Git-tf)

Upgrading to Team Foundation Server 2012

Testing enhancements in Visual Studio 2012 Update 1 and Update 2

“The Thrill of the Hunt” with Exploratory testing in Visual Studio 2012

I am hosting two of the sessions and I’ll be around to answer questions live on the day.

12:00  Being Agile with Team Foundation Server 2012

Team Foundation Server 2012 introduced a number of excellent features and capabilities to help enable your agile team to plan, manage and track their work. This session will introduce the tools and provide details on how to get the best out of them.

15:40  Testing enhancements in Visual Studio 2012 Update 1 and Update 2

Visual Studio 2012 Update 1 and the newly released Update 2 have introduced some exciting new features and functionality to the core testing tools.  This session will demonstrate the new web based Test Case Management, cross browser support for Coded UI Testing and Microsoft Test Manager improvements.

You can register for this event here and it’s worth doing even if you can’t make it on the day as we’ll follow up with resources and recordings of the sessions.

Cheers,

Richard

Tweet.