CS 45 — Infrastructure Setup

In this course, lab assignments will frequently ask you to make modifications to the Linux kernel. Thus, you will need to setup:

  • A (virtual) system where you can test your kernels, and

  • An environment to build Linux kernels.

This guide will walk you though both steps. Since they each need lots of storage space, you’ll need to work in /local, which is the local hard disk of the machine you’re using. Storing things on /local has two major implications that you need to account for:

  1. The data on /local is stored on a single disk (unlike your home directory, which is split across multiple disks for redundancy). When you’re done working, you should save your important lab files elsewhere (e.g., push them to github) to avoid data loss in the event of a disk failure. When you’re done working on your VM, you should shut it down nicely: "sudo shutdown -h now".

  2. The /local partition on each machine is only available on that machine. This means that if you want to move and work a different machine, you’ll need to set up a new VM at the new location. Alternatively, you could also copy your VM image from /local on one machine to /local on another with scp.

Environment — VM

For security and reliability reasons, you’ll be testing your kernels on a virtual machine. That way, if your kernel fails, you can simply reboot the VM and there’s no risk of damaging your CS files.

To get started, you’ll need to import a copy of the starter VM image:

  1. Run virtualbox in a terminal or open it from within your favorite graphical menu.

  2. Go to File → Preferences, set your "Default Machine Folder" to "/local", and then close the preferences window.

  3. Go to File → Import Appliance. Choose: /local/CS45-VM.ova and push next once.

  4. You should now be seeing "Appliance settings". Edit the name to include your username. For example, "CS45-kwebb".

  5. Click import and wait a minute for it to complete.

After you’ve completed these steps, you should see the your VM in the list of VMs available to start. Go ahead and turn it on. The username and password are both cs45.

Before going any further, you should use the passwd command to set a new password to protect your VM. Open a terminal and run passwd.

Kernel compilation — from scratch

At the start of each lab assignment, you’ll need to build a kernel from scratch. It will take approximately 10-15 minutes to compile, depending on which machine you use. At the end, you’ll get nice .deb packages that you can install.

While it’s annoying to have to wait 10-15 minutes to build your code, you’ll only need to perform this step once per assignment. After you’ve built the kernel this way once, you can switch to the incremental build method to only rebuild the parts that have changed.

Steps for building from scratch:

  1. Create a directory in /local to use for your kernel builds. I suggest /local/kernels-username, so for me, that would be /local/kernels-kwebb:

    mkdir /local/kernels-username  # Fill in your username
cd /local/kernels-username

  2. Download the Linux kernel sources. The apt package manager will make this easy:

    apt-get source linux

  3. Move into the newly created linux-version directory and download a configuration that will reduce the build time. Note that the version number you see might be slightly different!

    cd linux-4.15.0
wget https://www.cs.swarthmore.edu/~kwebb/cs45/s20/labs/.config

  4. By default, this will suffix your kernel’s name with -cs45-lab. You should change that to give your kernel a unique name so that you can identify it:

    make menuconfig

    Then navigate to the General Setup → local version setting. I would suggest naming it based on the current lab number you’re working on (e.g., -cs45-lab2). After setting the name, you can move the arrow to the right to exit (twice) and then confirm that you want to save your configuration changes.

  5. Prep the build by running some provided scripts:

    chmod a+x debian/rules
chmod a+x debian/scripts/*
chmod a+x debian/scripts/misc/*
fakeroot debian/rules clean

  6. Build it (this will take a while):

    export CONCURRENCY_LEVEL=6
time fakeroot make-kpkg --revision=1.0 kernel_image kernel_headers

You can adjust the revision number however you’d like. When it’s finished, you should have two new .deb files one directory up:

$ ls -l ../*.deb
-rw-r--r-- 1 kwebb users  9036852 Jan 21 12:39 ../linux-headers-4.15.17-cs45-lab2_1.0_amd64.deb
-rw-r--r-- 1 kwebb users 15250016 Jan 21 12:38 ../linux-image-4.15.17-cs45-lab2_1.0_amd64.deb

Installing a .deb packaged kernel on the VM

First, you’ll need to copy the .deb files to your VM. The easiest way to do this is to use scp, which works like cp for transferring files over a network. It uses encryption, so the 's' stands for 'secure'. From the VM:

# Move to the home directory.
cd

# Replace username with your username and hostname with the name of the machine you used to
# build your kernel.  Don't miss the trailing dot!
scp username@hostname:/local/kernels-username/*.deb .

You can now install the .deb packages using dpkg:

sudo dpkg -i package_name.deb

If you’ve already an earlier version of .deb packages with the same name, you’ll need to uninstall the old ones first:

sudo dpkg -r package_name

Reboot the VM to use the new kernel. After booting the system, run uname -a to confirm the kernel version that you’re running.

Kernel compilation — incremental changes

Because it’s so slow, building the entire kernel from scratch every time you make a change would be maddening. It’s a nice fallback in case something goes wrong, but otherwise you probably just want to rebuild the files that you’ve changed (plus anything that depends on those files).

If you make changes to a very important header file that gets included all over the place (e.g., sched.h), you’ll likely end up rebuilding a lot of the kernel.

From the linux-version directory, you can rebuild the kernel image with:

make -j6 bzImage

Installing an incrementally-built bzImage kernel:

You’ll need to copy two files from the kernel build directory to your VM. From the VM:

# Move to the home directory.
cd

# Replace username with your username and hostname with the name of the machine
# you used to build your kernel.

scp username@hostname:/local/kernels-username/linux-4.15.0/arch/x86_64/boot/bzImage .
#Don't miss the trailing dot.

scp username@hostname:/local/kernels-username/linux-4.15.0/System.map .
#Don't miss the trailing dot.

Now that the files are on the VM, you need to put them in /boot. You should overwrite the existing kernel that was created when you did the .deb file installation:

# Replace "whatever" with the name you gave to your kernel.
sudo mv -i bzImage /boot/vmlinuz-4.15.17-whatever
sudo mv -i System.map /boot/System.map-4.15.17-whatever
sudo update-grub

Reboot the VM to use the new kernel. After booting the system, run uname -a to confirm the kernel version that you’re running.

Booting Your Kernel

When your VM starts, it will present you with options for a few seconds. Choose "Advanced options for Ubuntu" to select which kernel you’d like to boot. I would suggest leaving the original kernel there as a fallback, just in case one of yours crashes on startup.

Tools for Working Remotely

While the purpose of this class isn’t really to teach tools, I think we have an opportunity here to help you all be more productive, and it’s useful to know how this stuff works before you head off into industry anyway.

Putting this all together, if you’d like to work from your personal machine, you can install virtualbox and copy the VM files to your laptop. Then, when you want to edit/build a kernel, you can use a combination of sshfs and ssh to edit files and then build. I’ve never used sshfs on Windows, but I know that several implementations exist. I’ve used it successfully on Mac OS X and Linux, where it should be relatively easy to install/setup.

Secure Shell (ssh)

ssh, the secure shell, is the workhorse. If you’re not familiar with ssh, you can use it to connect to a lab machine and remotely issue commands at a terminal. On Linux / OS X, you can run the ssh command from a terminal. On windows, you’ll need to use a terminal emulator like PuTTY. From a CS machine, you don’t need to specify a full name, you can just refer to it by spice name:

ssh basil

From another network, you’ll need to specify a fully-qualified domain name:

ssh basil.cs.swarthmore.edu

Secure Copy (scp)

If you need to move files between machines, you can use scp, the secure copy utility. It works like the normal cp command, where you do:

cp [source file location] [destination file location]

With scp though, one of the locations is allowed to be a remote machine that you can ssh to. Formatting the remote side is:

username@host:/path/to/file

If you want to copy an entire directory instead of just one file, you can pass the -r flag to do a recursive copy.

For example, if you want to move a VM from one machine to another you have two options:

  1. From the machine that has the VM, you could do:

    # "desthost" is the hostname of the machine you'd like to move the VM files to.
scp -r /local/CS45-username username@desthost:/local/

  2. From the destination machine, you could do:

    # "srchost" is the hostname of the machine you'd like to move the VM files from.
scp -r username@srchost:/local/CS45-username /local/

If you’d like to work on a laptop, you can install virtualbox on your laptop and copy the VM image to that.

The ssh File System (sshfs)

Even with a VM on your local machine, you’ll still need a place to edit/build your kernel. I would strongly suggest that you don’t try to move an entire kernel directory with scp. There are so many small files, it will take a very long time. Instead, you can use sshfs which allows you to mount a file system over an ssh connection.

In a nutshell, sshfs says "please make an ssh connection to machine X and use that connection to make it look like machine X’s files are available locally." So, if the machine with your kernel on it is occupied, you can mount its files remotely by doing:

sshfs username@host:/path/to/directory /path/to/mountpoint

The second parameter, /path/to/mountpoint, is the directory where you’d like to make the files from the remote machine appear to be available. You should use an empty directory, and on the CS cluster, it needs to be on /local (due to the way NFS works).

For example, if your kernel is on basil, but you’re working from eagle, you could execute the following on eagle:

# Replace 'username' with your actual username.
mkdir /local/sshfs-username
sshfs username@basil:/local/kernels-username /local/sshfs-username
cd /local/sshfs-username
ls

You should then see all of the kernel files as if you were on basil. You can edit them using your favorite editor with only a small delay when you save files (since they’re being transferred over the ssh connection).

DO NOT try to build your kernel over the sshfs file mount — it’ll be VERY slow. Instead, you can ssh to the machine with your kernel files and run the build process directly on that machine. In other words, keep two terminals open:

  1. A terminal that you for editing files locally over the sshfs mount.

  2. A terminal that’s ssh'ed to the machine that has your kernel files. You can issue remote build commands over ssh.