Integrating docker release management in TFS with webtests

TFS express 2017 provides free and fully integrated environment for continious integration and release management for docker windows containers. Post below only focuses on release management for docker containers via TFS and webtests produced by Visual Studio Enterprise edition.

My current setup of TFS build produces following artifacts available for release management.

  • docker-stack.yml

This file presented below provides information about build version of container image and in imagetag and additional information for running environment

  • tests folder with 3 files
    • local.testsettings
      • This file provides information which server webtest have to be running against
    • webtest1.webtest
      • This file provides actual steps which is tested
  • runtest.ps1
    • This file which performs 2 functions: waits for swarm manager to bring latest version of image up and then run webtests tests on those.

Release pipeline itself contains of 3 steps for each environment. Screenshot below for staging environment.


First step is pushing newly built image to staging docker swarm. This step is using built-in Docker command tasks in TFS and passing information about swarm location and registry connection via built in TFS properties along with docker-stack.yml file above.


Second step is powershell script named runtest.ps1above. Variables  are passed to script to specify which server needs to be checked against as well current version of new build version of docker image


Last step is publishing results of those tests


Below is screenshot what failure in webtest looks like. Along with failure also attached webtestresult file which you can open in Visual Studio to inspect details of what has failed.


Please note to enable test run you need to have mstest.exe installed on TFS agent as part of Visual Studio Enterprise 2017 installation. You can use Evaluation Version to install it. You need to start Visual Studio at least once under user which agent is running under to fix the issue below (if your agent runs as System service then you need to run Visual Studio at least one a System). Test will fail otherwise with File extension specified .webtest is not a valid test extension.

Another thing to note is that there is currently a bug in TFS 2017 U2 which does not properly pull results form TRX file and hence additional logic was added to powershell script to put results in correct location to Publish Test Results tag can find results.


2 thoughts on “Integrating docker release management in TFS with webtests

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s