I managed my evaluation, but it’s quite far from now (14 days), so I’m rather late on my blogs posts.

What happened during that long time? A lot of things.

PYTI summary

Since midterm evaluation, we started to works on a common summary to be published as a blog post and which will be an “introduction” to PYTI. We got something quite good, but finally with recent changes and some other reasons, it’s not ready to be published. We will try to give you access to this summary as soon as possible in order to have some feedback.

Works on common part

During all this time, we worked on the common part with yeswanth, called “manager”. The manager is responsible of checking if the VM has a snapshot, if not take one; uploading distributions (main one to be tested + its dependencies) and the recipe, launch the VM, get the result from VM and then rollback it. It asked a lot of discussions about where files will be located and how we configure our VM, but we finally end with theses decisions:

  • Do not provide a “base” VM, but provide a way to prepare a fresh installed VM to be used in PYTI context. It’s currently represented as a bash script that run only on Debian Like system (it should not require a lot of work on order to run on other Linux Distributions). This script is located here.
  • Define a manager without any path “hardcoded”.
  • Provide a way to launch VM in local for debug purpose. This script is located in the scripts directory in repository and named launch_vm.py. This script is not fully functionnal, partly because we encountered the vicious bug described below.

Understanding VirtualBox snapshot system

When we worked on the manager, we encounter a very vicious bug due to VBox snapshot system. I will describe the source of the bug and then I will list all of behavior we tried to understand without knowing the source of the bug.

First, we should have read the documentation about the snapshot system, really the doc is quite clear about the technical aspect of this system (documentation). When you take a snapshot of a VM, you will create a “differencing” image which will receive all write operations from the virtual machine. So if you make a snapshot of a VM, then you take a snapshot of it, all of writing operations you will do will not be located in the “source hd” image, but in another one, the “differencing image”. It’s this little detail that we missed, we imagine that each writing operations are done on the “source hd” image and when the rollback occurs, they’re reverted.

Now I will detail the list of strange behaviors we observe and I will let you imagine how we react by keeping in mind that we don’t master the snapshot system at this time:

  • Upload a file, download same file: Works.
  • Create a file in VM, try to download it: Fails, says that “File is not existing” but we can “see” it in the VM shell.
  • When we rollback the VM, files created in the VM are deleted, but files uploaded remained in the VM.

And the best for the end:

  • Upload a file, make some changes to the file in the VM, try to download it: Fails, download the file content which has been uploaded.

I really thought going crazy at some times, especially with last behavior. The problem was not so easy as we suspected the library we used for upload and download files (libguestfs), we are wrong of course.

To solve this problem, we will use two disk images, one for the OS, which will be normally concerned by the snapshot system and another one, just for Inputs (distributions + executon config) and Outputs (files generated by the VM).

“Lesson learnt”: Always RTFM.