Integrating docker release management in TFS with webtests

TFS express 2017 provides free and fully integrated environment for continious integration and release management for docker windows containers. Post below only focuses on release management for docker containers via TFS and webtests produced by Visual Studio Enterprise edition.

My current setup of TFS build produces following artifacts available for release management.

  • docker-stack.yml

This file presented below provides information about build version of container image and in imagetag and additional information for running environment

version: '3.2'
target: 80
published: 8080
protocol: tcp
mode: host
type: volume
source: webredirect
target: c:\inetpub\logs\logfiles\host
type: volume
source: webredirect-freb
target: c:\inetpub\logs\logfiles\freb
mode: global
condition: on-failure
delay: 5s
max_attempts: 3
window: 120s
parallelism: 1
constraints: [node.role != manager]
external: true
external: true
external: true

view raw
hosted with ❤ by GitHub

  • tests folder with 3 files
    • local.testsettings
      • This file provides information which server webtest have to be running against
<?xml version="1.0" encoding="UTF-8"?>
<TestSettings name="Local" id="8912d4fb-1ed0-4d91-98cf-787f3ffb50aa" xmlns="">
<Description>These are default test settings for a local test run.</Description>
<Deployment enabled="false" />
<TestTypeSpecific >
<WebTestRunConfiguration testTypeId="4e7599fa-5ecb-43e9-a887-cd63cf72d207" simulateThinkTimes="true" />
<Property name="TestSettingsUIType" value="LoadTest" />

view raw
hosted with ❤ by GitHub

    • webtest1.webtest
      • This file provides actual steps which is tested
<?xml version="1.0" encoding="utf-8"?>
<WebTest Name="WebTest1" Id="2b0301fa-6605-4d91-9c11-9798ac868ba5" Owner="" Priority="2147483647" Enabled="True" CssProjectStructure="" CssIteration="" Timeout="0" WorkItemIds="" xmlns="" Description="" CredentialUserName="" CredentialPassword="" PreAuthenticate="True" Proxy="default" StopOnError="True" RecordedResultFile="" ResultsLocale="">
<Request Method="GET" Guid="1ffa095e-785d-4935-ac00-ecdf45081b34" Version="1.1" Url="{{EndpointName}}/" ThinkTime="0" Timeout="300" ParseDependentRequests="False" FollowRedirects="False" RecordResult="True" Cache="False" ResponseTimeGoal="0" Encoding="utf-8" ExpectedHttpStatusCode="301" ExpectedResponseUrl="" ReportingName="" IgnoreHttpStatusCode="False" />
<Request Method="GET" Guid="1ffa095e-785d-4935-ac00-ecdf45081b34" Version="1.1" Url="{{EndpointName}}/mssplus.txt" ThinkTime="0" Timeout="300" ParseDependentRequests="False" FollowRedirects="False" RecordResult="True" Cache="False" ResponseTimeGoal="0" Encoding="utf-8" ExpectedHttpStatusCode="200" ExpectedResponseUrl="" ReportingName="" IgnoreHttpStatusCode="False" />
<ContextParameter Name="EndpointName" Value="" />

view raw
hosted with ❤ by GitHub

  • runtest.ps1
    • This file which performs 2 functions: waits for swarm manager to bring latest version of image up and then run webtests tests on those.
$tool = "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\MSTest.exe",
$path = $psscriptroot,
$item = "$path\webtest1.webtest",
$results = "$path\webtest.trx",
$testsettings = "$path\Local.testsettings",
$expectedReleaseHeader = "webredirect:20170726_1050",
$urlTest = ""
while ($true)
$null = Invoke-WebRequest $urltest UseBasicParsing MaximumRedirection 0 OutVariable SLheaderOut ErrorAction SilentlyContinue
Write-Output $_
if ($Slheaderout)
$SLheader = $Slheaderout.headers.SL
$Slheader = $null
if ($SLheader -ieq $expectedReleaseHeader )
Write-Output "Found $SLheader while expecting $expectedReleaseHeader"
Start-Sleep 5
Write-Output "Found $Slheader on $urlTest, can proceed with the rest of tests"
${env:Test.EndpointName} = $urlTest
& $tool /TestContainer:$item /resultsfile:$Results /testsettings:$testsettings /noisolation
#Temporary measure untill VSTS is fixed
New-Item Name "webtest" Path $psscriptroot ItemType Directory
move-item Path (join-path path $path Childpath (get-childitem Path $path Name "in" Directory Recurse)) Destination $path\webtest
#Install evaluation version of VS 2017 Enterprise

view raw
hosted with ❤ by GitHub

Release pipeline itself contains of 3 steps for each environment. Screenshot below for staging environment.


First step is pushing newly built image to staging docker swarm. This step is using built-in Docker command tasks in TFS and passing information about swarm location and registry connection via built in TFS properties along with docker-stack.yml file above.


Second step is powershell script named runtest.ps1above. Variables  are passed to script to specify which server needs to be checked against as well current version of new build version of docker image


Last step is publishing results of those tests


Below is screenshot what failure in webtest looks like. Along with failure also attached webtestresult file which you can open in Visual Studio to inspect details of what has failed.


Please note to enable test run you need to have mstest.exe installed on TFS agent as part of Visual Studio Enterprise 2017 installation. You can use Evaluation Version to install it. You need to start Visual Studio at least once under user which agent is running under to fix the issue below (if your agent runs as System service then you need to run Visual Studio at least one a System). Test will fail otherwise with File extension specified .webtest is not a valid test extension.

Another thing to note is that there is currently a bug in TFS 2017 U2 which does not properly pull results form TRX file and hence additional logic was added to powershell script to put results in correct location to Publish Test Results tag can find results.


Comparing windows containers CPU perfomance vs alternatives

When presenting containers solution to wider audience question of perfomance invariably comes to place (the same sort of discussion everybody had 10 years ago during virtualization craze). I decided to do unscientific test of running Windows containers vs alternatives. Specifically windows containers are compared against running the same application on physical hardware, inside Hyper-V VM on the same hardware as well as windows containers in process and hypervisolation modes.

You can check results by yourself as image is posted at artisticcheese\iis on docker hub.

Code which is used for testing is designed to return PI number calculates to certain number which is very CPU intensive operation. In my specific run I was calculating to 8000s place with total of 100 requests in multithreaded client.

Client code is below, which is powershell code relying on runspaces via PSParallel module

#requires -Modules PSParallel
Measure-command {1..100 | Invoke-Parallel -ScriptBlock
{Invoke-WebRequest http://localhost/service.svc/pi/8000 -UseBasicParsing}}

Below are results of running this tests against 4 different environments

Test environment Hyper-V Hardware Process Isolation HyperV isolation
96.47 90.12 90.77 91.4
97.12 90.13 90.86 90.24
97.87 89.27 90.49 91.3

Conclusion may be surprising or may be not but running Windows containers introduce virtually no overhead from CPU point of view on application perfomance.

Replacing ServiceMonitor.exe with IIS error log events in windows IIS container

ServiceMonitor.exe is used as default ENTRYPOINT entry official Microsoft IIS build. It does not seem to be doing much except for checking if service W3SVC is running. There is multitude of issues with this EXE based on github ( Bearing in mind very limited functionality that this EXE provides I decided it to replace with something more useful.
Starting with IIS 8.5 it’s possible to output IIS logs not only to log files but also to ETW events. You can consume those inside eventlog name called Microsoft-Windows-IIS-Logging/Logs which shall be enabled to receive those events. Steps below will allow to provide output of all errors being logged on your webserver to docker logs instead of default ENTRYPOINT of IIS image which does not provide any valuable information.
I also decided to start my images now in servercore instead of IIS since latter only installs WindowsFeature Web-Server and creates ENTRYPOINT for ServiceMonitor.exe. Neither of which is really necessary anyway.
To accomplish this 2 things would need to be changed in base Microsoft image.

  1. Enable IIS to log to ETW (in addition or insted of file logging)
  2. Enable log called Microsoft-IIS-Logging/logs

Step number 1 is accomplished by executing following powershell

Import-module WebAdministration
$splat = @{
    filter = "system.applicationHost/sites/siteDefaults/logFile"
    name =  "logTargetW3C"
    value = "File,ETW"
Set-WebConfigurationProperty @splat

Step 2 is below

$IISOpsLog = Get-WinEvent -ListLog Microsoft-IIS-Logging/logs
$IISOpsLog.IsEnabled = "true"

Both entries are made inside website_config.ps1 file in artifacts directory.

This setup will output ETW events to Microsoft-IIS-Logging/logs which will be repeatadly read in powershell script called entrypoint.ps1 below

$VerbosePreference = "ignore"
$sleep = 5
while ($true)
    $datediff = (New-TimeSpan -Seconds $sleep).TotalMilliseconds
    $filter = "*/System/TimeCreated[timediff(@SystemTime) <= $datediff] and *[EventData/Data[@Name='sc-status'] >'400']"
    Get-WinEvent -MaxEvents 10 -FilterXPath $filter -ProviderName "Microsoft-Windows-IIS-Logging" -ErrorAction SilentlyContinue | 
    Select-Object @{Name = "time"; e = {$_.Properties[2].value}}, @{Name = "VERB"; e = {$_.Properties[8].value}}, 
    @{Name = "ClientIP"; e = {$_.Properties[3].value}}, @{Name = "URI"; e = {$_.Properties[9].value}}, 
    @{Name = "Query"; e = {$_.Properties[10].value}}, @{Name = "Status"; e = {$_.Properties[11].value}}, 
    @{Name = "host"; e = {$_.Properties[21].value}} | Format-Table
    Start-Sleep $sleep

I restrict number of events returned by query at source based on time since last request as well as only for specific eventcodes which identifies Web Server errors (status codes > 400)

The last step is to put this script into ENTRYPOINT in DOCKERFILE

ENTRYPOINT powershell.exe C:\startup\entrypoint.ps1

Entire code base along with additional files is available on Github page (
Image on dockerhub is (

You can test functionality below

docker run -it artisticcheese/iis-admin

Issue request to non-existent file to local container


You will see output in stdout of container

time     VERB ClientIP     URI   Query Status host         
----     ---- --------     ---   ----- ------ ----         
18:28:08 GET /asda -        404