Testing Physical Machines with kitchen-static

Testing on Physical Machines with kitchen-static

This article shows how to work with Test Kitchen on physical machines using the kitchen-static Driver. If you need to deliver a product (bundle of server and software) instead of just configuration, some tasks cannot be run on virtual machines alone but need testing on actual hardware.

The need for testing on physical machines comes from a current project, where physical servers are part of a large industrial solution and are provisioned with Chef.

As with all projects, one should keep the Dev/Prod Parity Principle in mind, which states that development and production environment should be as similar as possible. For example, physical systems include components which are not available in your usual VMware- or AWS-based environment: Installation of vendor specific drivers, configuration of hardware components (ILO, BMC, …) or even setting certain types of Windows Network Teaming.

kitchen-static

Over a year ago, this kitchen driver was created to allow our first steps on HP Gen9 servers. It was easy enough, as Test Kitchen already comes with the proxy driver. Upon checking its source code it became clear that this driver was outdated and poorly maintained. As it was obvious that we are dealing with a total niche case here, the decision to maintain a custom driver was easy.

Installation of the driver is very straightforward as it has no external dependencies and is available on RubyGems:

gem install kitchen-static

Now, just change your kitchen.yml accordingly. I prefer having my platform specifics in a separate file kitchen.static.yml:

driver:
  name: static
  hostname: 198.51.100.14

If you have a machine at the given IP, your usual kitchen converge and kitchen verify commands work as intended.

Setup and Tear Down

As we are working with a physical machine which does not just vanish or pop into existence, we have to take care of not getting the machine messed up. After all, kitchen create and kitchen destroy just change a value in your local .kitchen directory and do not interact with the actual machine at all.

One decentralized solution are Kitchen Lifecycle Hooks. They have been added in Kitchen 1.23 and allow running local or remote commands on certain stages of Kitchen execution. We can use them to do our setup and tear down actions:

driver:
  name: static
  hostname: 198.51.100.14
  
lifecycle:
  post_create:
    - remote: setup.ps1
      environment:
        SOME_PARAMETER: value
  pre_destroy:
    - remote: Start-Process -FilePath C:\Windows\System32\Sysprep\Sysprep.exe -ArgumentList "/generalize /oobe /shutdown /quiet"

The logical execution steps are:

  • kitchen create: Add the IP address to the suite state file in .kitchen only
  • right after create: Run a script setup.ps1 which is present on the remote machine
  • … develop, test, retry …
  • right before destroy. Remotely trigger a SysPrep (reset to factory defaults for Windows
  • kitchen destroy: Remove the IP address from the state file

Of course, SysPrep takes time so you will now sit there and bite your nails for some minutes until you can retest.

Alternative for Restoring Baseline Configuration

While I have not benchmarked it, I think using System Restore Points instead of a full SysPrep might be quicker for Windows systems. These work using Restore-Computer -RestorePoint X in Powershell, where X is the ID of a Restore point you set up on initial deployment of the machine.

Linux-based Systems

For Ubuntu/Debian based systems there is no centralized, standard procedure to reset them to default as far as I know. You can try the tool Resetter here, which also has a command line interface.

Centralized Management of Machines

As soon as you have a bigger team involved in writing components for physical machines or have automated testing pipelines, the 1:1 approach of this post does not scale any more.

The logical solution is to have a central pool of machines and a mechanism to centrally hand out test machines, do queueing and standardized restore procedures. This functionality was introduced in version 0.10.0 of kitchen-static and will be the topic of another blog post.

Similar Posts You Might Enjoy

Mocking data in Test Kitchen (Part 2)

Going beyond the easier use case of mocking attributes and databags, we sometimes want to fake some data about the system itself. - by Thomas Heinen

Mocking data in Test Kitchen

The more complex your cookbooks, the bigger the need to supply some external information to your test machines. Passing specific attributes, values of databags or secrets for testing become necessary. We will go through these use cases and show how to mock the data in this post. - by Thomas Heinen

Update your Style in Test Kitchen

It is surprising how many resources on the Internet are carrying on outdated or deprecated information - the Chef ecosystem is no exception to this. While outdated style in Ruby files has been detected via cookstyle for a while, Test Kitchen files still have no sanity checks yet. Let’s see what changed in this short post. - by Thomas Heinen