Note: this is the older version of the virtualbox guide (used for lab 2), see the new virtual box guide for labs 3 and beyond.

Contents:

• What is VirtualBox?
• Setting up VirtualBox on the CS lab machines
• Running VirtualBox (staring virtualbox, staring an OS, shutting down an OS, exiting virtualbox)
• Compiling the Linux Kernel, and Installing and Booting a Linux kernel in VirtualBox
• Debugging the kernel
• ssh'ing and scp'ing to the VirtualBox VM from the host machine
• Kernel Debugging (printk and gdb)
• Troubleshooting VirtualBox and Linux

VirtualBox is x86 and AMD64/Intel64 virutalization software. You will use virtual box for labs that involve modifying and testing the Linux. Modifying the kernel source code and compiling it will be done directly on CS machines. You will install, boot, and test your modified kernel on the virtualbox virtual machine (VM). One benefit of using VirtualBox is that if your kernel crashes you do not crash the entire machine, just the VirtualBox VM.

The VirtualBox Homepage has more information about virtualbox and about using virtualbox.

This section describes what you need to do to set up virtualbox on a CS lab machine so that you and your project partner can start working. You first need:

1. A specific assigned CS machine on which you will run virtual box and do all your Linux kernel development.
2. A special shared user account that you and your partner will use for just this joint lab work.

Logging In

You and your partner will be assigned a user account that you can share for CS45 work, and a specific CS Lab machine on which you will compile your Linux kernel and run virtualbox.

You will store the virtualbox machine image and the Linux kernel source code in a subdirectory in /local on your assigned machine. Even though you can only run virtualbox on the machine on which you are assigned (since it is the only one, and should be the only one, with your virtualbox machine in /local), you can ssh into your CS lab machine from any other CS lab machine (or anywhere else) and run virtualbox remotely.

For example, if the PC's name is ginger, then you would type the following to connect (-X enables X forwarding, use -Y if ssh'ing from MAC-OSX):

ssh -X your_account_name@ginger 

The -X option enables X forwarding in a secure way.

If you ssh in from outside cs, I'd recommend running virtualbox in non-gui mode and not doing X forwarding, as it can be very slow.

Steps for setting up your VirtualBox environment

One of you or your partner:

1. ssh into the CS machine to which you have been assigned using your shared account (ex. ginger and cs45X), and choose a password for your shared account together so that both of you can remember it:

     
       % ssh -X cs45X@ginger
   

2. Make a working_dir directory in /local that you and your partner will share. For example, if my working_dir is named tia_n_pal:

     
       % mkdir /local/tia_n_pal 
       % chmod 700 /local/tia_n_pal
   

3. Copy over the virtualbox image files from /local/vm (a copy is also available on every CS in /scratch/vm, but it will take longer to copy over than the one in /local) into your /local/working_dir subdirectory:

       % cp -r /local/vm/ubuntu-12.04.3-STARTHERE  /local/tia_n_pal/. 
   

4. Now you are ready to run virtualbox.

Starting VirtualBox:

1. Log into the PC on which your virtualbox machine is located using your shared account (ex: ginger, cs45X):

       % ssh -X cs45X@ginger 
   

2. Start virtualbox:

       $ cd /local/you_n_partner/
       $ virtualbox 
       (ignore the "Error opening file for reading: Permission denied" error)
   

3. The very first time you run virtualbox, you need to explicitly add the VM image you want to run. To do this:
   From the Machine menu choose Add then browse through the files systems to select this file to open:

   /local/you_n_partner/ubuntu-12.04.3-STARTHERE/ubuntu-12.04.3-STARTHERE.vbox 
   

4. In the VM Mananger window, select the VM you want to run (there is only one choice) and then choose the start icon to start it running.

   The starting point .vbox VM contains a single linux kernel to boot. As you add more kernels, you can select the particular kernel you want to boot from the grub menu using the arrow keys and Enter.

running VirtualBox in non-gui mode

You can also run virtualbox in non-graphics mode. This is useful when remotely connecting from outside of the cs subnet, or when you want to start up virtualbox once, and then remotely log in over a period of time. In the later case, you will need to run virtualbox within a screen session as well as in non-graphics mode. Here are the steps:

$ screen    # start a screen session
$ vboxheadless -s "ubuntu-12.04.3-STARTHERE"  # run virtualbox in non-gui mode
Cntrl-a d          # detach from the screen session (machine keeps running)

note: this will always boot the default kernel (specified in the grub menu) (and it takes about 1 minute to boot). If the default kernel is not the one you want to boot, you can change the default kernel:

(1) in /etc/default/grub, set GRUB_DEFAULT to the number kernel entry you want:
    GRUB_DEFAULT=0    # the default kernel is the one in entry 0
                      # change it to be the entry you want as the default (0,1,2,...)

   you can list the kernels in which entry:
    grep menuentry /boot/grub/grub.cfg

(2) then run:
    sudo update-grub

Starting an OS inside of VirtualBox:

When virtualbox is run in gui mode, and you first boot up the system, you will see the grub menu with a list of kernels to boot. Use the arrow keys to scroll through the list and then hit the Enter key to choose your kernel. If you don't select one, after 10 seconds the default kernel (at the top of the list) is booted. If you run in non-gui mode, the default kernel boots.

After the kernel boots in your VM you can log into the console window. However, it will be easier to work on your virtual machine by ssh'ing into it. To do this specify port number 10022 and the CSmachine on which you are running virtualbox. There is one user account (swatcs) with the starting point VM, its password (swatcs.45):

ssh -p 10022 swatcs@ginger   

The virtualbox image is set up with a single user account: swatcs. The password is swatcs.45. You can change any of your user account passwords, by running the passwd command. You can also add new users to the system using the adduser command.

Shutting down or rooting an OS:

To shut down your kernel:

$ sudo sync
$ sudo sync
$ sudo halt -p     # halt and power down

To reboot your kernel:

$ sudo sync
$ sudo sync
$ sudo reboot

Shutting down VirtualBox:

First shut down your kernel, then exit virtualbox using the menu options. If you run in non-gui mode, just use CNTRL-C to exit vboxheadless.

You will compile and build your changes to the Linux kernel on your CS machine in /local/, and then install your built kernel on the virtualbox VM.

Getting a copy of the Linux Source

These steps only need to be done once (or whenever you want to grab a fresh copy of the code):

1. copy the kernel source into your /local directory:

    $ cp /local/vm/linux-2.6.32.44.tar.bz2 /local/you_n_partner/. 
   

2. unzip and untar the file:

    $ cd /local/you_n_partner
    $ tar xjf linux-2.6.32.44.tar.bz2 
   

3. copy the starting point config file into .config in linux-2.6.32.44 and run make menuconfig:

    $ cd linux-2.6.32.44 
    $ cp /local/vm/config-working-2.6.32.44 .config
    $ make menuconfig 	# just choose Exit with the arrow keys
   

4. follow Option 1 below for building the kernel the first time.

Building the kernel

Building kernel image (and kernel header) packages

1. Set this environment variable once in the shell you are compiling in (it will use a parallel compile and result in a faster total compile time):

   export CONCURRENCY_LEVEL=4    # anything higher than 4 doesn't help 
   

2. Do this the first time, and maybe other builds, but you do not always have to do this on every build (it is like a 'make clean'):

   # NOTE: make-kpkg clean wipes out everything that has been compiled before.
   #  For the very first build you need to do this step, but for subsequent 
   #  builds you may want to not do it, and only if package builds are 
   #  failing in odd ways try doing a make-kpkg clean to re-build 
   #  everything from scratch ("from scratch" takes ~5 minutes)
   $ cd /local/you_n_partner/linux-2.6.32.44
   $ fakeroot make-kpkg clean
   

3. build the kernel packages you want to build (typically kernel_headers and/or usually kernel_image).

   # NOTE: change "whatever" after --append-to-version to something more 
   #       meaningful to you.  For example, I might append lab2 to my 
   #       kernel version with my lab 2 solution 
   #       (a clean build will take about 5 minutes):
   $  fakeroot make-kpkg --initrd --revision=1.0 --append-to-version -whatever kernel_image
   
   # if you add new kernel header files, or change existing ones, you need 
   # to build a kernel headers package that contains your changed files too:
   $  fakeroot make-kpkg --initrd --revision=1.0 --append-to-version -whatever  kernel_headers
   

   make-kpkg creates files named:

   ../linux-image-2.6.32.44-whatever-cs45_1.0_amd64.deb 
   ../linux-headers-2.6.32.44-whatever-cs45_1.0_amd64.deb
   

   You also can list both kernel_image and kernel_headers in a single make-kpkg command line.

NOTE: if this is a rebuild of a previous package, then you will need to remove the previous package before installing the new one:

# on your VM
dpkg -i linux-image-2.6.32.44-whatever-cs45

Installing and booting your kernel

Installing kernel packages:

# (a) from your CS machine, copy the kernel packages to your VM:
#     (I'm using ginger as an example CS machine name)
$ scp -P 10022 linux-image-2.6.32.44-whatever-cs45_1.0_amd64.deb swatcs@ginger:.
$ scp -P 10022 linux-headers-2.6.32.44-whatever-cs45_1.0_amd64.deb swatcs@ginger:.

# (b) Next, from within your VM, install the packages:
#     from root's home directory (or wherever you scp the files) 
$ sudo dpkg -i linux-image-2.6.32.44-whatever-cs45_1.0_amd64.deb
$ sudo dpkg -i linux-headers-2.6.32.44-whatever-cs45_1.0_amd64.deb 
note: ignore this error message when installing the headers package (it is not really an error):
Error! Your kernel headers for 2.6.34-whatever cannot be found.

# (c) reboot your VM and choose your new kernel from the grub menu
$  sudo sync; sudo sync; sudo reboot

# (d) login and verify that the new version of your kernel rebooted 
$ uname -a

# note: If this is a re-install of a kernel package 
# that you have already installed (i.e. the same -whatever flag as an 
# installed kernel package), you need to first remove the 
# old package(s), before you do the dpkg -i of the new ones: 
$ dpkg -r linux-image-2.6.32.44-whatever-cs45 

# Use the -I option to dpkg to list info about the package file, 
# including its name (used in the -r option)
$ dpkg -I linux-image-2.6.32.44-whatever-cs45_1.0_amd64.deb 

An Important Note about Modifying Kernel Source Files:
We do not back-up files in each machine's /local directory. As a result, you should periodically copy files into your private cs45 subdirectory in /home, which we do back-up ( Retrieving Lost Files).

I suggest that you use git to do this:

1. one of you should create a shared bare git repository (and share it with three users: you, your partner, and your CS45 shared account user).
2. You and your partner should clone a copy in your private /home/you/cs45 subdirectories.
3. Your shared CS45 account user should clone a copy into /local on your assigned machine.

As you work, periodically add, commit, and push changes to the linux kernel files you modify (and also to any user-level programs you write as part of a lab assignment).

Make sure to include in your git repo only the linux source and header files that you modify for the current lab assignment, and not the entire linux kernel source tree, which will eat up your account quota. These are the files that you will submit for each lab assignment, and the ones that you cannot recover. If you accidentally delete other kernel files (ones you aren't changing), you can always get a copy of them from the linux source starting point. If you lose files you have modified, then can then get a back-up from your git repo.

Use ssh to log in to your VM from the assigned CS lab machine from which you are running virtualbox:

$ ssh -p 10022 swatcs@ginger

Use scp to transfer files between your VM and your CS machine. Here are some examples (and yes, it is -P for scp and -p for ssh):

$ scp -P 10022  file.c swatcs@ginger:.
$ scp -P 10022  *.c swatcs@ginger:.          # copy a bunch of .c files
$ scp -P 10022  -r dirname swatcs@ginger:.   # recurively copy a directory

From your VM you can scp to your CS user account. Here are some examples:

$ scp file username@machinename:/home/username/cs45/labs/lab2/.
$ scp file cs45username@machinename:/local/you_n_partner/lab2/.

You do not have valgrind to help you debug at the kernel level, but you can use the kernel's kgdb support to remotely attach gdb to your running kernel and debug it. You can also add debugging statements to your kernel using printk.

printk

A useful way to see what your kernel code is doing is to add debugging printk statements to print out some execution state as as it runs. printk is similar to printf, except remember that floating point values are not supported at kernel level.

printk output goes to the console and to log files on your VM in /var/log/kern.log and /var/log/syslog. You can open them in vim to printk output. Also, running tail -f on one of these files will let you see printk output as your kernel runs:

   
$ tail -f /var/log/kern.log

If your kernel crashes, it is likely that some of the printk output has not yet been written to the log files. As a result, you may want to jot down the last printk output on the console before rebooting. Also, be very careful about where you put printk calls; you don't want them on a code path that is executed many times per second.

printk output should also show up on the console window. If not, run this command, and then it should:

sudo dmesg -n 7

kgdb

You can use the linux kernel's built-in support for kgdb to attach gdb to your running kernel and debug it. The default config file you copied over already builds the kernel image with kgdb support enabled. You need to follow some steps on boot to connect gdb running on the host machine (your CS machine) to the linux kernel VM running in VirtualBox. (this link contains some nice screen dumps showing some of these steps: KGDB with VirtualBox).

Steps to attach gdb to your VM:

1. Run virtualbox, but do not boot a VM (or halt your VM if already running), and then configure the serial port:

   Click on your virtual machine instance in the VirtualBox Manager window (your VM should not be running), and choose Settings. Select the "Serial Ports" tab, choose Port 1, and enter the following and the choose OK:

   (1) Check the "Enable Serial Port" box
   (2) Choose COM1 for "Port Number"
   (3) Choose HostPipe for "Port Mode"
   (4) Check the "Create Pipe" box
   (5) Enter /local/me_and_pal/serial in the "Port/File Path" box  
       (where /local/me_and_pal/ is the name of your subdirectory /local on
        on your machine in which your virtualbox image and linux source are) 
   

2. Start the boot of your VM, and at the grub menu choose the kernel version you want, and then enter e to edit its information. Scroll down to the linux line and append the following to it (and these are zeros not ohs):

   
   kgdboc=ttyS0,115200 kgdbwait
   
   # The result will look something like:
   linux /boot/vmlinux/.... \
   ...  ro kgdboc=ttySO,115200 kgdbwait
   

   Then hit F10 to continue to boot. Your VM will continue booting but not finish, printing a message:

   kgdb: Waiting for connection from remote gdb..." message
   

3. At this point, in a terminal on the host machine (your CS machine) run:

   socat -d -d /local/me_and_pal/serial PTY:
   
   # you should see output like this:
   2014/01/31 14:13:24 socat[27748] N opening connection to AF=1 "/local/me_n_pal/serial"
   2014/01/31 14:13:24 socat[27748] N successfully connected from local address AF=1 "qp\xEE\x7E"
   2014/01/31 14:13:24 socat[27748] N successfully connected via 
   2014/01/31 14:13:24 socat[27748] N PTY is /dev/pts/16
   2014/01/31 14:13:24 socat[27748] N starting data transfer loop with FDs [3,3] and [4,4]
   

   Note: the /dev/pts/16 part will likely be different, and use the one that yours lists as the target in gdb.

   Do not terminate socat; it needs to be running in the background to use the pseudo terminal to connect gdb.

4. in another terminal on the host machine (your lab machine) run gdb and attacke to the remote target:

   $ cd /local/me_and_pal/linux-2.6.32.44/
   $ gdb ./vmlinux
   ...
   ...
   Reading symbols from /local/me_and_pal/linux-2.6.32.44/vmlinux...done.
   
   (gdb) target remote /dev/pts/16       # or whatever the number is from socat 
   Remote debugging using /dev/pts/16
   kgdb_breakpoint () at kernel/kgdb.c:1718
   1718            wmb(); /* Sync point after breakpoint */
   

   At this point you can use gdb commands to debug your kernel code: set breakpoints in your kernel functions, step and next through the execution of code, print variables, ...

I'll post answers to virtualbox and Linux questions here.