Using IPAM Static IP Addresses with Cloudbase-init and vRealize Automation

Using IPAM Static IP Addresses with Cloudbase-init and vRealize Automation

I have been eating, breathing and sleeping VMware Cloud Management and Automation lately. I have learned quite a bit, over the last 6 months, about automation concepts, designing for cloud-agnostic deployments and even Microsoft Windows deployment practices and issues. In doing so, I can across some really cool ideas and products that help along the way.

I am trying to incorporate as much functionality as possible, while staying within the confines of the vRealize stack. I am a big fan of using a product to it’s fullest potential. With that said, the issue I’m going to describe, and provide a solution, I opted not to take the easy road, and bolt on other products such as Ansible or Terraform. Because of that, I spent quite a bit of time on this single issue, and I have finally found a resolution, which I aim to document in this post.

For those of you that have found this post from a Google search, I assume you are having the same issue I did: you are trying to use vRealize Automation to deploy a Windows Server VM using Cloudbase-init to customize the deployment, while assigning a static IP through the built-in vRA IPAM functionality. If that’s the case, you are in the right place.

To give you some background on this “issue”, there is a situation that arises when you use IPAM-assigned static IPs during the deployment of a VM to vSphere and using cloud-init/cloudbase-init to customize the deployment. The issue stems from the way that vRA assigns the static IP to the VM during the deployment, and when the cloud-config kicks in. When vRA deploys a VM to vSphere, it helps you out by setting the IP and hostname of the VM, for you. That is automation at its basics. The problem with this, is that in doing so, vRA generates a dynamic Customization Specification, and applies it to the VM deployment. If you know anything about how a Customization Specification works, then you know that it employs Sysprep on Windows Machines. When the VM first boots up, Sysprep takes over, applies the configuration of the hostname and static IP, then reboots the server. This happens faster than cloudbase-init has the ability to run it’s customizations, and the automatic reboot by Sysprep interrupts the cloud-config. This is the high-level rundown of why the “issue” exists.

Now, to discuss how I managed to get to the resolution I came up with. I want to make sure that I give a shout-out to the folks/blogs that helped me get to the where I am, because they allowed me to be able to keep a little of my hair. In all the research I did, I came across a few blogs that helped point me in the right direction. The first, was a blog by Maher AlAsfar. This blog got me thinking that if there is a way to disable guest cusomization in Linux, then there has to be a way to do the same thing in Windows VMs. The problem with that, is I can’t find anything in any config file, that does this. After literally two months of searching, putting this aside and coming back to it, I finally found something that worked for me.

Here, you will find documentation that talks about an option, in vRA Blueprints, that will help solve our problem. This document is dated April 30, 2021, but claims to be for vRA 8.2, though 8.2 was released months before that date. The link should take you to page 334, which discusses this issue, and gives several examples of how to work around it. The key to all of this, is the option “customizeGuestOs”. This option actually allows us to tell vRA to not build and apply a dynamic Customization Specification. Well this solves all my problems.

This particular document calls out the option to be used for Linux VMs, but you can bet I tried it on a Windows VM. And guess what: IT WORKED!! So, now I’ve created a cloud template (blueprint in 8.2 and older), and have deployed a VM, and no dynamic customization specification has been applied. The problem that I have now, is that the OVF Service for Cloudbase-init doesn’t support static network assignments. So what do we do?! We script!!

I ended up writing a Powershell script to assign the static IP through cloud-config. I found this to be the easiest route to take; mostly because I had already come across another issue where using multi-part cloud-configs solve some issues. Here is the blog by Dana Gertsch, Kovarus Senior Consultant, that helped me figure out the issue of trying to mix regular cloud-config with inline scripts, along with this reddit post, which links to some documentation and another good post.

Here is my Cloud_Machine section for a cloud agnostic blueprint:

  Cloud_Machine_1:
    type: Cloud.Machine
    properties:
      name: '${input.hostname}'
      image: Windows Server 2019 - Cloudbase-init
      flavor: Windows-small
      customizeGuestOs: false
      networks:
        - network: '${resource.Cloud_Network_1.id}'
          assignment: static
      constraints:
        - tag: 'Cloud:vSphere'
        - tag: 'Environment:Production'
      storage:
        constraints:
          - tag: 'Environment:Production'
      cloudConfig: |
        Content-Type: multipart/mixed; boundary="==NewPart=="
        MIME-Version: 1.0

        --==NewPart==
        Content-Type: text/cloud-config; charset="us-ascii"
        MIME-Version: 1.0
        Content-Transfer-Encoding: 7bit
        Content-Disposition: attachment; filename="cloud-config"

        write_files:
          - content: |
              cloudbase-init - ${input.hostname}
              Cloud - ${resource.Cloud_Machine_1.account}
              IP Address - ${resource.Cloud_Machine_1.networks.address[0]}
              Netmask - ${resource.Cloud_Machine_1.networks.netmask[0]}
              Prefix Length - ${resource.Cloud_Network_1.prefixLength}
              Gateway - ${resource.Cloud_Machine_1.networks.gateway[0]}
              DNS - ${replace(replace(to_string(resource.Cloud_Machine_1.networks.dns[0]),"]",")"),"[","(")}
            path: c:\temp\test.txt

        set_hostname: ${input.hostname}

        --==NewPart==
        Content-Type: text/x-shellscript; charset="us-ascii"
        MIME-Version: 1.0
        Content-Transfer-Encoding: 7bit
        Content-Disposition: attachment; filename="VM-init.ps1"

        #ps1_sysnative
        $adapter = Get-NetAdapter | ? { $_.Name -eq "Ethernet0" }
 
        $adapter | Remove-NetIpAddress -Confirm:$false
        $adapter | Remove-NetRoute -Confirm:$false
        $adapter | New-NetIpAddress -IPAddress ${resource.Cloud_Machine_1.networks.address[0]} -PrefixLength ${resource.Cloud_Network_1.prefixLength} -DefaultGateway ${resource.Cloud_Machine_1.networks.gateway[0]}
        $adapter | Set-DnsClientServerAddress -ServerAddresses ${replace(replace(to_string(resource.Cloud_Machine_1.networks.dns[0]),"]",")"),"[","(")}

        sleep 15

        if ( "${input.domain}" -ne "Workgroup" ) {
            $creds = New-Object System.Management.Automation.PSCredential "${input.username}",(ConvertTo-SecureString -String "${secret.Password}" -AsPlainText -Force)
            Add-Computer -domainName ${input.domain} -Credential $creds -Restart
        }

A couple of things to note in this code:

• My template WAS NOT sysprep’d. I expect this won’t make much of a difference, but it could. Maybe when I get more time, I might test that.
• Once you disable the Guest OS Customization Script (GOSC), you must apply all your customizations on your own.
• I used the multipart cloud configuration to make things a little easier for me. Cloudbase-init really only supports cloud-config and x-shellscript as useful MIME types, so keep that in mind if you try to use any other MIME types that Cloud-Init supports.
• My template had a static IP when I packaged it, so in an effort to make this universal to all circumstances, I setup the script to remove any existing IPs before adding the new IP.
• I had to add a 15 second sleep to allow time for the IP address to apply completely before I could run the Add-Computer command.
• The x-shellscript above gets dumped into the cloudbase-init logs, which includes your password. I’m working to find a better solution for this, but in the meantime, I have a script that deletes the logs after cloudbase-init completes its customizations.