Rocky and Palette eXtended Kubernetes

This guide teaches you how to use the CAPI Image Builder tool to create a custom Rocky Linux image with Palette eXtended Kubernetes (PXK) for VMware vSphere and use the image to create a cluster profile. You can use either a Rocky Linux boot ISO or an existing Rocky Linux VM to create your image.

tech preview

This is a Tech Preview feature and is subject to change. Do not use this feature in production workloads.

Prerequisites

Rocky ISO

• Access to the VMware vSphere environment, including credentials and permission to create virtual machines.

• An existing Linux device used to execute commands and build your Rocky image. This device must have the following resources available and the following software installed:

  • 4 CPUs
  • 8 GB of RAM
  • 50 GB of free disk space
  • Docker or Podman
  • curl
  • (Optional) Any custom Bash scripts (.sh files) that you want to execute when creating your Rocky image. Custom scripts are supported beginning with CAPI Image Builder version 4.6.23.

Rocky VM

• Access to the VMware vSphere environment, including credentials and permission to create virtual machines.

• An existing Linux device used to execute commands and build your Rocky image. This device must have the following resources available and the following software installed:

  • 4 CPUs
  • 8 GB of RAM
  • 50 GB of free disk space
  • Docker or Podman
  • curl
  • (Optional) Any custom Bash scripts (.sh files) that you want to execute when creating your Rocky image. Custom scripts are supported beginning with CAPI Image Builder version 4.6.23.

• An existing VM with an OS of Rocky Linux 8 or 9 installed. This VM will be used as the base of your image and must meet the following requirements:

  • The following tools installed:

    • conntrack-tools
    • cloud-init
    • cloud-utils-growpart
    • iptables
    • python2-pip
    • python3

  • A user of builder with a password of builder. This is required by the vsphere-clone-builder. The builder user must be granted passwordless sudo privileges.

    builder user and password privileges

    1. On your Rocky Linux VM, add a builder user.

       sudo useradd builder

    2. Set the password for the builder user to builder.

       echo 'builder:builder' | sudo chpasswd

    3. Assign passwordless sudo privileges to the builder user and assign the appropriate permissions.

       echo 'builder ALL=(ALL) NOPASSWD: ALL' | sudo tee /etc/sudoers.d/builder
       sudo chmod 0440 /etc/sudoers.d/builder

  • SSH password authentication enabled in /etc/ssh/sshd_config by setting PasswordAuthentication to yes. You must either restart sshd or reboot your system for the changes to take effect.

  • SSH password authentication enabled for cloud-init by setting ssh_pwauth to true. This is required to prevent cloud-init from overwriting PasswordAuthentication yes in /etc/ssh/sshd_config when booting the cloned VM. We recommend creating a separate file that explicitly sets ssh_pwauth: true.

    sudo tee /etc/cloud/cloud.cfg.d/99-enable-ssh-pwauth.cfg << EOF
    ssh_pwauth: true
    EOF

  • IPv4 packet forwarding enabled.

  • firewalld disabled.

    sudo systemctl disable --now firewalld

  • /tmp mounted to execute binaries and scripts.

    Check /tmp status

    1. Check the mount status of /tmp. Look for a status of noexec.

       mount | grep '/tmp'
       Example output

       tmpfs on /tmp type tmpfs (rw,nosuid,nodev,noexec,relatime,size=2G)
       tip

       If you receive an error, where /tmp is not displayed in the mount output, it is likely because it is a regular directory on the filesystem and not a separate mount.

       Issue the following command to confirm the mount point of /tmp. If the Mounted on location is /, no action is required.

       df --human-readable /tmp

       Example output

       Filesystem           Size  Used Avail Use% Mounted on
       /dev/mapper/rl-root   70G  3.7G   67G   6% /

    2. If /tmp has a status of noexec, use your preferred text editor to edit the file /etc/fstab and set /tmp to exec.

       vi /etc/fstab
       Example output

       /dev/mapper/rl-root                         /                       xfs     defaults                            0 0
       UUID=3b068723-b40a-4c10-ac6d-00271cd4d3a4   /boot                   xfs     defaults                            0 0
       UUID=F867-A7CE                              /boot/efi               vfat    umask=0077,shortname=winnt          0 2
       /dev/mapper/rl-home                         /home                   xfs     defaults                            0 0
       /dev/mapper/rl-swap                         none                    swap    defaults                            0 0
       tmpfs                                       /tmp                    tmpfs   defaults,nosuid,nodev,exec,size=2G  0 0

    3. Remount all filesystems in /etc/fstab.

       sudo mount --all

    4. Confirm the mount status of /tmp is set to exec.

       mount | grep '/tmp'
       Example output

       tmpfs on /tmp type tmpfs (rw,nosuid,nodev,exec,relatime,size=2G)

  • If your system has been hardened using a Security Technical Implementation Guide (STIG) policy, you may need to remediate the following:

    • SELinux may prevent binaries from executing, including cloud-init scripts. We recommend setting the SELinux status to permissive or disabled until the image building process is complete.

      Check SELinux status

      1. Check the status of SELinux.

         getenforce
         Example output

         Enforcing

      2. If the status is Enforcing, use your preferred text editor to open the SELinux config file and set SELINUX to either permissive or disabled.

         vi /etc/selinux/config
         Example output

         # This file controls the state of SELinux on the system.
         # SELINUX= can take one of these three values:
         #     enforcing - SELinux security policy is enforced.
         #     permissive - SELinux prints warnings instead of enforcing.
         #     disabled - No SELinux policy is loaded.
         SELINUX=permissive
         # SELINUXTYPE= can take one of these three values:
         #     targeted - Targeted processes are protected,
         #     minimum - Modification of targeted policy. Only selected processes are protected.
         #     mls - Multi Level Security protection.
         SELINUXTYPE=targeted

    • fapolicyd may prevent certain applications from executing, such as containerd-shim-runc-v2. We recommend disabling fapolicyd until the image building process is complete.

      sudo systemctl disable --now fapolicyd

  • A snapshot of your VM created once all other prerequisites are met. This is required by the vsphere-clone-builder.

Build Custom Image

Rocky ISO

1. Open a terminal session on your Linux machine and set your CAPI Image Builder version tag as a variable. This guide uses version 4.6.24 as an example. Refer to the CAPI Image Builder Downloads page for the latest version.

   CAPI_IMAGE_BUILDER_VERSION=<capi-image-builder-version-tag>
   echo CAPI Image Builder version: $CAPI_IMAGE_BUILDER_VERSION
   Example output

   CAPI Image Builder version: v4.6.24

2. Download the CAPI Image Builder image.

   Docker

   docker pull us-docker.pkg.dev/palette-images/palette/imagebuilder/capi-builder:$CAPI_IMAGE_BUILDER_VERSION

   Podman

   podman pull us-docker.pkg.dev/palette-images/palette/imagebuilder/capi-builder:$CAPI_IMAGE_BUILDER_VERSION

3. Create an output directory to store the image files and set the required permissions.

   mkdir /home/$USER/output
   chmod a+rwx /home/$USER/output

4. Navigate to the output directory.

   cd /home/$USER/output

5. Download the Rocky Linux ISO file into the output directory. Ensure you download a x86_64-dvd.iso file and not a x86_64-boot.iso file.

   This guide uses Rocky 8 as an example. Refer to the Configuration Reference page for details on supported operating systems.

   curl https://download.rockylinux.org/pub/rocky/8/isos/x86_64/Rocky-8-latest-x86_64-dvd.iso --output Rocky-8-latest-x86_64-dvd.iso

6. Calculate the SHA256 checksum for the Rocky ISO you downloaded. The calculation might take a few minutes. Save the output, as you will need it later.

   sha256sum Rocky-8-latest-x86_64-dvd.iso

   The output should be similar to the sample output displayed below.

   Example Rocky SHA

   642ada8a49dbeca8cca6543b31196019ee3d649a0163b5db0e646c7409364eeb  Rocky-8-latest-x86_64-dvd.iso

7. Download the imageconfig template file.

   curl https://software.spectrocloud.com/tools/capi-image-builder/imageconfig --output imageconfig

8. Open the imageconfig template file in an editor of your choice and fill in the required parameters. The imageconfig file is the file used to personalize the base CAPI image for your cluster, which you can alter to fit your needs. This includes specifying the OS type, Kubernetes version, whether the image should be FIPS compliant, and more.

   The following example configuration configures a Rocky 8 CAPI image from a Rocky ISO using the SHA256 checksum of the Rocky ISO from step 6 of this guide. Replace all VMware-related placeholders in the Define Vmware infra details section with values from your VMware vSphere environment.

   For a complete list of parameters, refer to the Configuration Reference page. Additionally, refer to the Compatibility Matrix for a list of supported Kubernetes versions and their corresponding dependencies.

    # Define the OS type and version here
    # os_version=rhel-8 | rhel-9 | rockylinux-8 | rockylinux-9
    # image_type=standard | fips
    os_version=rockylinux-8
    image_type=standard
   
    # Define the image name
    # image_name=<Final Image Name to create>
    image_name=rocky-8
   
    # Define the Cloud type
    # cloud_type=vmware
    cloud_type=vmware
   
    # Define the Component Versions
    #
    # containerd crictl and cni version update should be done
    #   only if the images are available in the upstream repositories
    k8s_version=1.30.4
    cni_version=1.3.0
    containerd_version=1.7.13
    crictl_version=1.28.0
   
    # Define RHEL subscription credentials(if $image_type=rhel)
    # used while image creation to use package manager
    #rhel_subscription-user=
    #rhel_subscription_pass=
   
    # Define ISO url(if image is rhel or rockylinux)
    iso_name=Rocky-8-latest-x86_64-dvd.iso
    iso_checksum=<iso-checksum>
   
    # Define AWS infra details
    aws_access_key=
    aws_secret_key=
   
    # Define Vmware infra details
    vcenter_server=<vcenter-server>
    vcenter_user=<vcenter-user>
    vcenter_password=<vcenter-password>
    vcenter_datacenter=<vcenter-datacenter>
    vcenter_datastore=<vcenter-datastore>
    vcenter_network=<vcenter-network>
    vcenter_folder=<vcenter-folder>
    vcenter_cluster=<vcenter-cluster>
    vcenter_resource_pool=<vcenter-resource-pool>
   
    # Optional: for OVA based builds
    vcenter_template=
   
    # Define Azure infra details
    azure_client_id=
    azure_client_secret=
    azure_subscription_id=
    azure_location=
    azure_storage_account=
    azure_resource_group=
   
    # Define GCE infra details
    google_app_creds=
    gcp_project_id=
   
    # Airgap Configuration
    airgap=false
    airgap_ip=""
    k8s_rpm_key=
    k8s_rpm_server=
    containerd_url=
    crictl_url=
    k8s_container_reg=
    cert_url=
   tip

   To build a FIPS-compliant image, keep the image_type set to fips.

   Once you are finished making changes, save and exit the file.

9. (Optional) You can add custom Bash scripts (.sh files) to run before or after the build process. This feature is available beginning with CAPI Image Builder version 4.6.23. If any scripts are found in the relevant directories, they are copied to an Ansible playbook. If you do not want to add custom scripts, skip this step.

   Add Pre- and Post-Install Bash Scripts

   1. In the output directory, create the directories custom_scripts/pre and custom_scripts/post.

      mkdir -p custom_scripts/pre custom_scripts/post

   2. Move any scripts that you want to be executed before the build process to the pre directory. Move any scripts that you want to be executed after the build process to the post directory. Ensure the scripts are executable.

      Below is an example of moving a pre-install script to the appropriate pre directory and making it executable.

      Example of moving a script and modifying permissions

      mv sample-script.sh custom_scripts/pre/sample-script.sh
      chmod +x custom_scripts/pre/sample-script.sh

10. Issue the command below to start the CAPI Image Builder container and assign the container ID to the BUILD_ID variable. The tool will create and configure a VM with Dynamic Host Configuration Protocol (DHCP) in your VMware vSphere environment using the image_name defined in imageconfig. The tool will then generate a Rocky image from the VM and save it to the output directory.

    Docker

    BUILD_ID=$(docker run --net=host --volume /home/$USER/output:/home/imagebuilder/output  --detach  us-docker.pkg.dev/palette-images/palette/imagebuilder/capi-builder:$CAPI_IMAGE_BUILDER_VERSION)

    Podman

    BUILD_ID=$(podman run --net=host --volume /home/$USER/output:/home/imagebuilder/output  --detach  us-docker.pkg.dev/palette-images/palette/imagebuilder/capi-builder:$CAPI_IMAGE_BUILDER_VERSION)

    If you need the VM to use static IP placement instead of DHCP, follow the steps described below.

    CAPI Image Builder with Static IP Placement

    1. Download the Rocky 8 ks.cfg file from the Image Builder GitHub repository directly into the output folder.

       curl --location https://github.com/kubernetes-sigs/image-builder/raw/main/images/capi/packer/ova/linux/rockylinux/http/8/ks.cfg.tmpl --output ks.cfg

    2. Open the ks.cfg file in an editor of your choice. Locate and replace the network line network --bootproto=dhcp --onboot=on --ipv6=auto --activate --hostname=capv.vm with the configuration below.

       network --bootproto=static --ip=<vcenter-static-ip-address> --netmask=<vcenter-netmask> --gateway=<vcenter-gateway> --nameserver=<vcenter-nameserver>

       Replace <vcenter-static-ip-address> with a valid IP address from your VMware vSphere environment and <vcenter-netmask>, <vcenter-gateway>, and <vcenter-nameserver> with the correct values from your VMware vSphere environment. The <vcenter-netmask> parameter must be specified in dotted decimal notation, for example, --netmask=255.255.255.0.

       Once you are finished making changes, save and exit the file.

    3. Issue the command below to start the CAPI Image Builder container and assign the container ID to the BUILD_ID variable. The tool will use the imageconfig file to create and configure a VM with static IP placement in your VMware vSphere environment.

       Docker

       BUILD_ID=$(docker run --net=host --volume /home/$USER/output:/home/imagebuilder/output  --detach  us-docker.pkg.dev/palette-images/palette/imagebuilder/capi-builder:$CAPI_IMAGE_BUILDER_VERSION)

       Podman

       BUILD_ID=$(podman run --net=host --volume /home/$USER/output:/home/imagebuilder/output  --detach  us-docker.pkg.dev/palette-images/palette/imagebuilder/capi-builder:$CAPI_IMAGE_BUILDER_VERSION)

11. Execute the following command to view the CAPI Image Builder container logs and monitor the build progress. It may take a few minutes for the logs to start being displayed, and the build takes several minutes to complete. If you added any custom scripts in step 9, the output will be displayed in the build log.

    Docker

    docker logs --follow $BUILD_ID

    Podman

    podman logs --follow $BUILD_ID

12. Once the build is complete, the Rocky CAPI image is downloaded to the output directory as the image_name specified in the imageconfig file. Issue the following command to confirm that the build files are present in the output directory.

    ls -l <image_name>
    Example output

    -rw-r--r-- 1 ubuntu ubuntu       1203 Nov 18 02:48 packer-manifest.json
    -rw-r--r-- 1 ubuntu ubuntu 3571576320 Nov 18 02:48 rocky-8-disk-0.vmdk
    -rw-r--r-- 1 ubuntu ubuntu       9507 Nov 18 02:48 rocky-8-fips.ovf
    -rw-r--r-- 1 ubuntu ubuntu        212 Nov 18 02:48 rockylinux-8-kube-v1.30.4.mf
    -rw-r--r-- 1 ubuntu ubuntu 3571630080 Nov 18 02:49 rockylinux-8-kube-v1.30.4.ova
    -rw-r--r-- 1 ubuntu ubuntu         64 Nov 18 02:49 rockylinux-8-kube-v1.30.4.ova.sha256
    -rw-r--r-- 1 ubuntu ubuntu      41044 Nov 18 02:48 rockylinux-8-kube-v1.30.4.ovf

13. Locate the new Rocky image VM in your VMware vSphere environment. Right-click the VM and select Clone > Clone to Template.

    tip

    Once the image is built, you can connect to the image via SSH. The following steps are based on guidance from the Image Builder Book.

    Connect to image VM with SSH

    1. On a machine with govc installed and configured with your VMware vSphere credentials, clone the Kubernetes Image Builder repository.

    2. Navigate to the capi directory of the Kubernetes Image Builder repository.

       cd ./image-builder/images/capi/

    3. Run the Kubernetes Image Builder image-govc-cloudinit.sh script and pass in the image_name of your Rocky image VM as specified in the imageconfig file. This creates a snapshot of the image and updates it with the data located in the cloudinit directory. Ensure the VM is off before running the command.

       ./hack/image-govc-cloudinit.sh <image_name>
       Example output

       image-govc-cloudinit: creating snapshot 'new'
       image-govc-cloudinit: initializing cloud-init data
       image-govc-cloudinit: creating snapshot 'cloudinit'

    4. Set read-write permissions for the id_rsa.capi file.

       chmod 600 cloudinit/id_rsa.capi

    5. Power on the Rocky image VM.

    6. Connect to the VM via SSH using your id_rsa.capi key. Replace <vm-ip> with the IP of your Rocky image VM.

       ssh -i cloudinit/id_rsa.capi capv@<vm-ip>

14. Enter a VM template name, choose a location for the template, and select Next.

    info

    The name and location do not have to match those defined in the imageconfig file. The same applies to the remaining locations and resources specified in the following steps.

15. Choose a compute resource and select Next.

16. Choose a storage location and select Next.

17. Review your template configurations and select Finish to convert the VM into a Rocky image template that you can reference when creating your cluster profile.

Rocky VM

1. Open a terminal session on your Linux machine and set your CAPI Image Builder version tag as a variable. This guide uses version 4.6.24 as an example. Refer to the CAPI Image Builder Downloads page for the latest version.

   CAPI_IMAGE_BUILDER_VERSION=<capi-image-builder-version-tag>
   echo CAPI Image Builder version: $CAPI_IMAGE_BUILDER_VERSION
   Example output

   CAPI Image Builder version: v4.6.24

2. Download the CAPI Image Builder image.

   Docker

   docker pull us-docker.pkg.dev/palette-images/palette/imagebuilder/capi-builder:$CAPI_IMAGE_BUILDER_VERSION

   Podman

   podman pull us-docker.pkg.dev/palette-images/palette/imagebuilder/capi-builder:$CAPI_IMAGE_BUILDER_VERSION

3. Create an output directory to store the image files and set the required permissions.

   mkdir /home/$USER/output
   chmod a+rwx /home/$USER/output

4. Navigate to the output directory.

   cd /home/$USER/output

5. Download the imageconfig template file.

   curl https://software.spectrocloud.com/tools/capi-image-builder/imageconfig --output imageconfig

6. Open the imageconfig template file in an editor of your choice and fill in the required parameters. The imageconfig file is the file used to personalize the base CAPI image for your cluster, which you can alter to fit your needs. This includes specifying the OS type, Kubernetes version, whether the image should be FIPS compliant, and more.

   The following example configuration configures a Rocky 8 CAPI image from an existing Rocky 8 VM in VMware vSphere. Replace all VMware-related placeholders in the Define Vmware infra details section with values from your VMware vSphere environment. Additionally, for vcenter_template, enter the full datacenter path to the Rocky VM that you want to use as a base for your CAPI image.

   For a complete list of parameters, refer to the Configuration Reference page. Additionally, refer to the Compatibility Matrix for a list of supported Kubernetes versions and their corresponding dependencies.

    # Define the OS type and version here
    # os_version=rhel-8 | rhel-9 | rockylinux-8 | rockylinux-9
    # image_type=standard | fips
    os_version=rockylinux-8
    image_type=standard
   
    # Define the image name
    # image_name=<Final Image Name to create>
    image_name=rocky-8
   
    # Define the Cloud type
    # cloud_type=vmware
    cloud_type=vmware
   
    # Define the Component Versions
    #
    # containerd crictl and cni version update should be done
    #   only if the images are available in the upstream repositories
    k8s_version=1.30.4
    cni_version=1.3.0
    containerd_version=1.7.13
    crictl_version=1.28.0
   
    # Define RHEL subscription credentials(if $image_type=rhel)
    # used while image creation to use package manager
    #rhel_subscription-user=
    #rhel_subscription_pass=
   
    # Define ISO url(if image is rhel or rockylinux)
    iso_name=
    iso_checksum=
   
    # Define AWS infra details
    aws_access_key=
    aws_secret_key=
   
    # Define Vmware infra details
    vcenter_server=<vcenter-server>
    vcenter_user=<vcenter-user>
    vcenter_password=<vcenter-password>
    vcenter_datacenter=<vcenter-datacenter>
    vcenter_datastore=<vcenter-datastore>
    vcenter_network=<vcenter-network>
    vcenter_folder=<vcenter-folder>
    vcenter_cluster=<vcenter-cluster>
    vcenter_resource_pool=<vcenter-resource-pool>
   
    # Optional: for OVA based builds
    vcenter_template=<vcenter-datacenter-path-to-VM>
   
    # Define Azure infra details
    azure_client_id=
    azure_client_secret=
    azure_subscription_id=
    azure_location=
    azure_storage_account=
    azure_resource_group=
   
    # Define GCE infra details
    google_app_creds=
    gcp_project_id=
   
    # Airgap Configuration
    airgap=false
    airgap_ip=""
    k8s_rpm_key=
    k8s_rpm_server=
    containerd_url=
    crictl_url=
    k8s_container_reg=
    cert_url=
   tip

   To build a FIPS-compliant image, keep the image_type set to fips.

   Once you are finished making changes, save and exit the file.

7. (Optional) You can add custom Bash scripts (.sh files) to run before or after the build process. This feature is available beginning with CAPI Image Builder version 4.6.23. If any scripts are found in the relevant directories, they are copied to an Ansible playbook. If you do not want to add custom scripts, skip this step.

   Add Pre- and Post-Install Bash Scripts

   1. In the output directory, create the directories custom_scripts/pre and custom_scripts/post.

      mkdir -p custom_scripts/pre custom_scripts/post

   2. Move any scripts that you want to be executed before the build process to the pre directory. Move any scripts that you want to be executed after the build process to the post directory. Ensure the scripts are executable.

      Below is an example of moving a pre-install script to the appropriate pre directory and making it executable.

      Example of moving a script and modifying permissions

      mv sample-script.sh custom_scripts/pre/sample-script.sh
      chmod +x custom_scripts/pre/sample-script.sh

8. Issue the commands below to start the CAPI Image Builder container and assign the container ID to the BUILD_ID variable.

   The tool will create and configure a VM with Dynamic Host Configuration Protocol (DHCP) in your VMware vSphere environment using the image_name defined in imageconfig. The tool will then generate a Rocky image from the VM and save it to the output directory.

   info

   The following commands provide a workaround for an existing issue by temporarily suspending the build and removing an erroneous task that checks for Red Hat Subscription Management (RHSM) credentials. For the final command, replace <os_version> with the Rocky os_version referenced in the imageconfig file (rockylinux-8 or rocklinux-9).

   Docker

   BUILD_ID=$(docker run --net=host --volume /home/$USER/output:/home/imagebuilder/output  --detach  us-docker.pkg.dev/palette-images/palette/imagebuilder/capi-builder:$CAPI_IMAGE_BUILDER_VERSION)
   BUILD_ID=$(docker run --net=host --volume /home/$USER/output:/home/imagebuilder/output --detach us-docker.pkg.dev/palette-images/palette/imagebuilder/capi-builder:$CAPI_IMAGE_BUILDER_VERSION sleep infinity)
   docker exec $BUILD_ID bash --command 'sed --in-place "/Fail if RHSM_USER or RHSM_PASS/,/lookup.*RHSM_PASS.*length == 0/d" /home/imagebuilder/ansible/roles/setup/tasks/redhat.yml'
   docker exec $BUILD_ID make build-node-ova-vsphere-clone-<os_version>

   Podman

   BUILD_ID=$(podman run --net=host --volume /home/$USER/output:/home/imagebuilder/output  --detach  us-docker.pkg.dev/palette-images/palette/imagebuilder/capi-builder:$CAPI_IMAGE_BUILDER_VERSION)
   BUILD_ID=$(podman run --net=host --volume /home/$USER/output:/home/imagebuilder/output --detach us-docker.pkg.dev/palette-images/palette/imagebuilder/capi-builder:$CAPI_IMAGE_BUILDER_VERSION sleep infinity)
   docker exec $BUILD_ID bash --command 'sed --in-place "/Fail if RHSM_USER or RHSM_PASS/,/lookup.*RHSM_PASS.*length == 0/d" /home/imagebuilder/ansible/roles/setup/tasks/redhat.yml'
   docker exec $BUILD_ID make build-node-ova-vsphere-clone-<os_version>

   If you need the VM to use static IP placement instead of DHCP, follow the steps described below.

   CAPI Image Builder with Static IP Placement

   1. Download the Rocky 8 ks.cfg file from the Image Builder GitHub repository directly into the output folder.

      curl --location https://github.com/kubernetes-sigs/image-builder/raw/main/images/capi/packer/ova/linux/rockylinux/http/8/ks.cfg.tmpl --output ks.cfg

   2. Open the ks.cfg file in an editor of your choice. Locate and replace the network line network --bootproto=dhcp --onboot=on --ipv6=auto --activate --hostname=capv.vm with the configuration below.

      network --bootproto=static --ip=<vcenter-static-ip-address> --netmask=<vcenter-netmask> --gateway=<vcenter-gateway> --nameserver=<vcenter-nameserver>

      Replace <vcenter-static-ip-address> with a valid IP address from your VMware vSphere environment and <vcenter-netmask>, <vcenter-gateway>, and <vcenter-nameserver> with the correct values from your VMware vSphere environment. The <vcenter-netmask> parameter must be specified in dotted decimal notation, for example, --netmask=255.255.255.0.

      Once you are finished making changes, save and exit the file.

   3. Issue the commands below to start the CAPI Image Builder container and assign the container ID to the BUILD_ID variable. The tool will use the imageconfig file to create and configure a VM with static IP placement in your VMware vSphere environment.

      Note that the following commands provide a workaround for an existing issue by temporarily suspending the build and removing an erroneous task that checks for Red Hat Subscription Management (RHSM) credentials. For the final command, replace <os_version> with the Rocky os_version referenced in the imageconfig file (rockylinux-8 or rocklinux-9).

      Docker

         BUILD_ID=$(docker run --net=host --volume /home/$USER/output:/home/imagebuilder/output  --detach  us-docker.pkg.dev/palette-images/palette/imagebuilder/capi-builder:$CAPI_IMAGE_BUILDER_VERSION)
         BUILD_ID=$(docker run --net=host --volume /home/$USER/output:/home/imagebuilder/output --detach us-docker.pkg.dev/palette-images/palette/imagebuilder/capi-builder:$CAPI_IMAGE_BUILDER_VERSION sleep infinity)
         docker exec $BUILD_ID bash --command 'sed --in-place "/Fail if RHSM_USER or RHSM_PASS/,/lookup.*RHSM_PASS.*length == 0/d" /home/imagebuilder/ansible/roles/setup/tasks/redhat.yml'
         docker exec $BUILD_ID make build-node-ova-vsphere-clone-<os_version>

      Podman

         BUILD_ID=$(podman run --net=host --volume /home/$USER/output:/home/imagebuilder/output  --detach  us-docker.pkg.dev/palette-images/palette/imagebuilder/capi-builder:$CAPI_IMAGE_BUILDER_VERSION)
         BUILD_ID=$(podman run --net=host --volume /home/$USER/output:/home/imagebuilder/output --detach us-docker.pkg.dev/palette-images/palette/imagebuilder/capi-builder:$CAPI_IMAGE_BUILDER_VERSION sleep infinity)
         docker exec $BUILD_ID bash --command 'sed --in-place "/Fail if RHSM_USER or RHSM_PASS/,/lookup.*RHSM_PASS.*length == 0/d" /home/imagebuilder/ansible/roles/setup/tasks/redhat.yml'
         docker exec $BUILD_ID make build-node-ova-vsphere-clone-<os_version>

9. Execute the following command to view the CAPI Image Builder container logs and monitor the build progress. It may take a few minutes for the logs to start being displayed, and the build takes several minutes to complete. If you added any custom scripts in step 7, the output will be displayed in the build log.

   Docker

   docker logs --follow $BUILD_ID

   Podman

   podman logs --follow $BUILD_ID

10. Once the build is complete, the Rocky CAPI image is downloaded to the output directory as the image_name specified in the imageconfig file. Issue the following command to confirm that the build files are present in the output directory.

    ls -l <image_name>
    Example output

    -rw-r--r-- 1 ubuntu ubuntu       1203 Nov 18 02:48 packer-manifest.json
    -rw-r--r-- 1 ubuntu ubuntu 3571576320 Nov 18 02:48 rocky-8-disk-0.vmdk
    -rw-r--r-- 1 ubuntu ubuntu       9507 Nov 18 02:48 rocky-8-fips.ovf
    -rw-r--r-- 1 ubuntu ubuntu        212 Nov 18 02:48 rockylinux-8-kube-v1.30.4.mf
    -rw-r--r-- 1 ubuntu ubuntu 3571630080 Nov 18 02:49 rockylinux-8-kube-v1.30.4.ova
    -rw-r--r-- 1 ubuntu ubuntu         64 Nov 18 02:49 rockylinux-8-kube-v1.30.4.ova.sha256
    -rw-r--r-- 1 ubuntu ubuntu      41044 Nov 18 02:48 rockylinux-8-kube-v1.30.4.ovf

11. Locate the new Rocky image VM in your VMware vSphere environment. Right-click the VM and select Clone > Clone to Template.

    tip

    Once the image is built, you can connect to the image via SSH. The following steps are based on guidance from the Image Builder Book.

    Connect to image VM with SSH

    1. On a machine with govc installed and configured with your VMware vSphere credentials, clone the Kubernetes Image Builder repository.

    2. Navigate to the capi directory of the Kubernetes Image Builder repository.

       cd ./image-builder/images/capi/

    3. Run the Kubernetes Image Builder image-govc-cloudinit.sh script and pass in the image_name of your Rocky image VM as specified in the imageconfig file. This creates a snapshot of the image and updates it with the data located in the cloudinit directory. Ensure the VM is off before running the command.

       ./hack/image-govc-cloudinit.sh <image_name>
       Example output

       image-govc-cloudinit: creating snapshot 'new'
       image-govc-cloudinit: initializing cloud-init data
       image-govc-cloudinit: creating snapshot 'cloudinit'

    4. Set read-write permissions for the id_rsa.capi file.

       chmod 600 cloudinit/id_rsa.capi

    5. Power on the Rocky image VM.

    6. Connect to the VM via SSH using your id_rsa.capi key. Replace <vm-ip> with the IP of your Rocky image VM.

       ssh -i cloudinit/id_rsa.capi capv@<vm-ip>

12. Enter a VM template name, choose a location for the template, and select Next.

    info

    The name and location do not have to match those defined in the imageconfig file. The same applies to the remaining locations and resources specified in the following steps.

13. Choose a compute resource and select Next.

14. Choose a storage location and select Next.

15. Review your template configurations and select Finish to convert the VM into a Rocky image template that you can reference while creating your cluster profile.

Create Cluster Profile

The Rocky image is now built and available in the VMware vSphere environment. You can use it to create a cluster profile and deploy a VMware host cluster.

1. Log in to Palette.

2. From the left main menu, select Profiles > Add Cluster Profile.

3. In the Basic Information section, assign the cluster profile a Name, brief Description, and Tags. Choose Full or Infrastructure for the profile Type, and select Next.

4. In the Cloud Type section, choose VMware vSphere, and select Next.

5. Select the Bring Your Own OS (BYOOS) pack and provide the following values in the YAML configuration editor. Proceed to the Next layer when finished.

   Field	Description	Example
   osImageOverride	The path to your Rocky Linux image template in your VMware vSphere environment.	/Datacenter/vm/sp-docs/rockylinux-8-kube-v1.30.4
   osName	The type of operating system used in your image.	rockylinux
   osVersion	The version of your operating system. Enter 8 or 9 depending on the Rocky Linux os_version referenced in the imageconfig file.	8

   Example YAML configuration

   pack:
     osImageOverride: "/Datacenter/vm/sp-docs/rockylinux-8-kube-v1.30.4"
     osName: "rockylinux"
     osVersion: "8"

6. Select the Palette eXtended Kubernetes (PXK) pack. Ensure the Pack Version matches the k8s_version specified in the imageconfig file. Proceed to the Next layer.

7. Complete the remaining profile layers, making any changes necessary. When finished, select Finish Configuration to create your cluster profile. For additional information on creating cluster profiles, refer to our Create an Infrastructure Profile and Create a Full Profile guides.

Next Steps

After you have created an OS image with CAPI Image Builder and have it referenced in a cluster profile, you can deploy a VMware host cluster using the created cluster profile. Refer to the Deploy App Workloads with a PCG tutorial for instructions on deploying a VMware host cluster.