TFS express 2017 provides free and fully integrated environment for continious integration and release management for docker windows containers. Post below only focuses on release management for docker containers via TFS and webtests produced by Visual Studio Enterprise edition.
My current setup of TFS build produces following artifacts available for release management.
This file presented below provides information about build version of container image and in
imagetag and additional information for running environment
- tests folder with 3 files
- This file provides information which server webtest have to be running against
- This file provides actual steps which is tested
- This file which performs 2 functions: waits for swarm manager to bring latest version of image up and then run webtests tests on those.
Release pipeline itself contains of 3 steps for each environment. Screenshot below for staging environment.
First step is pushing newly built image to staging docker swarm. This step is using built-in Docker command tasks in TFS and passing information about swarm location and registry connection via built in TFS properties along with docker-stack.yml file above.
Second step is powershell script named
runtest.ps1above. Variables are passed to script to specify which server needs to be checked against as well current version of new build version of docker image
Last step is publishing results of those tests
Below is screenshot what failure in webtest looks like. Along with failure also attached
webtestresult file which you can open in Visual Studio to inspect details of what has failed.
Please note to enable test run you need to have mstest.exe installed on TFS agent as part of Visual Studio Enterprise 2017 installation. You can use Evaluation Version to install it. You need to start Visual Studio at least once under user which agent is running under to fix the issue below (if your agent runs as
System service then you need to run Visual Studio at least one a
System). Test will fail otherwise with
File extension specified .webtest is not a valid test extension.
Another thing to note is that there is currently a bug in TFS 2017 U2 which does not properly pull results form TRX file and hence additional logic was added to powershell script to put results in correct location to Publish Test Results tag can find results.