Circumventing Windows containers limitation on volume mounts

Windows containers can not map files into container (only directory mounting is allowed). This commonly required to provide sensitive data to container are run time (example are connection strings in web.config or any other values needed for application which shall not end up in container image).

Common solution to this is to provide environment variable which indicate the location of secret configuration file and then override ENTRYPOINT in image to copy/process information before application starts. Something like below which check presence of environment variable ConfigPath and if it’s present then copy that file dynamically at runtime to override base image configuration file.

if($env:ConfigPath){
    Copy-Item -path $env:ConfigPath -Destination c:\inetpub\wwwroot -Force
}

This approach has major drawback – any update to volume data will not be visible to application since this script only runs at container start up.

To curcumvent this limitation you can create symbolic link instead pointing to desired target location. This approach will still allow you to have base configuration file in place for local debuggin etc but it will be overriden at runtime with symlink to desired secret.

Steps and verification are below (based on Azure AKS implementation but this will work accross any kubernetes or other orchestration or local solution)

  1. Create AKS cluster with windows node pool
  2. Create deployment and secret based on this file
apiVersion: apps/v1
kind: Deployment
metadata:
  name: secret-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: secret
  template:
    metadata:
      labels:
        app: secret
    spec:
      containers:
        - name: secret-container
          image: mcr.microsoft.com/dotnet/framework/aspnet:4.8-windowsservercore-ltsc2019
          env:
            - name: ConfigLocation
              value: c:\secret\web.config
          command: ["powershell"]
          args:
            - 'if ($env:ConfigLocation) {New-Item -Path C:\inetpub\wwwroot\Web.config -ItemType SymbolicLink -Value $env:ConfigLocation -force -Verbose}; & "C:\ServiceMonitor.exe" "w3svc"'
          volumeMounts:
            - name: secret
              mountPath: "secret"
      volumes:
        - name: secret
          secret:
            secretName: secretconfig
      nodeSelector:
        kubernetes.io/os: windows
      tolerations:
        - key: kubernetes.io/os
          operator: Equal
          value: windows
          effect: NoSchedule
---
apiVersion: v1
kind: Secret
metadata:
  name: secretconfig
type: Opaque
stringData:
  web.config: |
    <?xml version="1.0" encoding="utf-8"?>
    <configuration>
      <location path="." inheritInChildApplications="false">
        <system.webServer>
          <handlers>
            <add name="aspNetCore" path="*" verb="*" modules="AspNetCoreModuleV2" resourceType="Unspecified" />
          </handlers>
          <aspNetCore processPath=".\MyApp.exe"
                      stdoutLogEnabled="false"
                      stdoutLogFile=".\logs\stdout"
                      hostingModel="inprocess" />
        </system.webServer>
      </location>
    </configuration>

Deployment above creates secret with contents of desired web.config file, volume mapping into container and then overrides ENTRYPOINT to create symbolic link to secret.

Inside Kubernetes in fact similar concept is used to map secret name to a value. Executing below shows that actual secret web.config is actually a link to ..\data\web.config with directory ..data is in turn symbolink to dynamically named folder.

PS C:\Users\artis> kubectl exec deploy/secret-deployment powershell "get-childitem c:\secret | select name, linkType, target"
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.

Name                            LinkType     Target
----                            --------     ------
..2021_05_02_17_05_33.369337563
..data                          SymbolicLink {..2021_05_02_17_05_33.369337563}
web.config                      SymbolicLink {..data\web.config}

You can also verify that web.config is in fact mapped to desired value in container

PS C:\Users\artis> kubectl exec deploy/secret-deployment powershell "get-content c:\inetpub\wwwroot\web.config"
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <location path="." inheritInChildApplications="false">
    <system.webServer>
      <handlers>
        <add name="aspNetCore" path="*" verb="*" modules="AspNetCoreModuleV2" resourceType="Unspecified" />
      </handlers>
      <aspNetCore processPath=".\MyApp.exe"
                  stdoutLogEnabled="false"
                  stdoutLogFile=".\logs\stdout"
                  hostingModel="inprocess" />
    </system.webServer>
  </location>
</configuration>

Last thing to check if dynamic mapping of secret into running container. Changing slightly secret configuration file and redeploying it in fact shows new version of configuration item and in turn means IIS will pick up a change and will reload application domain.

PS C:\Users\artis> kubectl exec deploy/secret-deployment powershell "get-content c:\inetpub\wwwroot\web.config"
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
<?xml version="1.0" encoding="utf-8"?>
<configuration>
  New version
  </location>
</configuration>

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s