CMSC 421: Principles of Operating Systems

Homework 1: Building a Custom Linux Kernel

This homework is due on Monday, September 14, at 11:59:59 PM (Eastern daylight time). You must use the submit to turn in your homework like so: submit cs421_jtang hw1 hw1.tar.bz2

All of your homework assignments must run under 64-bit Linux, specifically Ubuntu 14.04 LTS ("Trusty Tahr"), and all projects will involve making changes to the Linux kernel. In addition, all assignments will be performed inside of a Virtual Machine for various reasons (uniformity of results, for instance). In order to do the later assignments in this course, you must become familiar with running Linux on your own computer. For projects, you will be compiling the kernel inside the VM and submitting kernel source code. In this homework, you well become familiar with the procedures of installing and updating the kernel inside a VM.

This homework is divided into several parts that will help you perform the following objectives:

• Becoming familiar with the tools used to build a custom Linux kernel image.
• Installing Linux and building a custom kernel.
• Modifying the Linux kernel.
• Creating a git commit.

Hardware and Software Requirements

Flash Drives and USB Hard Drives

If you choose to work in the lab, you will need an external USB hard drive or flash drive for development as well as for backups. You will need to have at least 30 GiB of free space on the drive to store your VM image on it, so a 32-GiB or larger drive is highly recommended. Also, you will need to format the drive to filesystems that supports files of size > 4GiB. Many flash drives come pre-formatted as FAT32, which does not support files with a size >= 4GiB. FAT32 will not work for this reason! You should format your drive as either NTFS (if you will be using Windows with the drive as well) or a Linux filesystem such as ext4.

Linux on VirtualBox

In order to aid development of assignemnts in this course, we will be running the submissions under virtualization. Virtualization allows us to run a Virtual Machine on an actual physical machine. In this way, we can run a second guest operating system inside the regular host operating system. To do the assignments in this class, we assume you will have access to a relatively modern PC that can run VirtualBox. VirtualBox requires an x86 CPU with a decent amount of RAM, at least 4 GiB. In addition, it would be advantageous if the CPU on your host machine supported the x86 virtualization extensions (VT-x for Intel processors, or AMD-V for AMD processors). For more information about hardware and software requirements for VirtualBox, please consult the VirtualBox website. All assignments are expected to run in 64-bit mode on VirtualBox.

VirtualBox is available on the machines in the ITE 240 lab under both Linux and Windows. You can also download it for free from https://www.virtualbox.org/wiki/Downloads to run under your own Windows, Mac, or even Linux host operating system.

For the purposes of assignments in this course, we will be using the x86 (64-bit) version of the Ubuntu 14.04 Linux distribution. In addition, the custom kernels that are built in this class will be based on the Linux kernel version 4.1 (the latest long-term stable kernel).

Installing Linux

Perform the following tasks to create the environment that will be used to complete assignments in this course. These instructions are based upon a WikiHow article.

1. Install VirtualBox on your computer. This particular assignment has been tested using version 4.3.30. If you are feeling especially adventuresome, try the brand new version 5.0.
2. For optimal VirtualBox performance, you need to enable virtualization on your computer. The exact steps varies based by computer. See this general guide. (Tablets and convertibles may not be capable at all.)
3. Download the Ubuntu 14.04.1 LTS x86 LiveCD. Make sure you select 64-bit from the drop-down menu. You can also make a donation to Ubuntu, if you so desire.
4. Scan through the Ubuntu release notes. Probably a lot of it will be incomprehensible. These notes will make more sense as you gain more experience in your Linux career.
5. Create a Virtual Machine for your Ubuntu 14.04 installation.
   1. Using VirtualBox, create a new virtual machine. You must give it a name, such as 421VM. Give ample memory (at least 2 GiB) and virtual hard disk space (at least 30 GiB).
   2. Your newly created virtual machine requires additional configuration. Set the number of processors to 2, and increase video memory to 64 MiB.
   3. Set the boot device to the Ubuntu ISO image you downloaded above. Power on the virtual machine to boot into the Ubuntu LiveCD.
   4. Run the Ubuntu installer. This will take a while, as that the installer will download additional files from the Internet.
   5. After installation, shut down your VM. You should now have an installation of Ubuntu on the VM that does not require the LiveCD image.
   6. Remove the LiveCD from the list of disks in the VM.
   7. Reboot the VM, and ensure that it boots into Linux properly.
6. Use Firefox to ensure that you can connect to the Internet within your VM.
7. Open a Terminal by clicking on the Ubuntu icon in the top-left corner. In the search box, type in terminal and hit enter to launch the Terminal program. On the left panel, right-click the Terminal icon and select Lock to Launcher.
8. Update the packages installed on the VM by executing the following command in the Terminal:

   sudo apt-get update && sudo apt-get upgrade

   You will need to enter your password to run the operation as root.
9. After updating, reboot the VM, to ensure that all updates complete before proceeding. The kernel may be updated by apt, and you must reboot to have the kernel updates applied.
10. Install the VirtualBox Guest Additions. While running your Linux VM, choose Insert Guest Additions CD from the Devices menu. Run the installer and reboot for a third time. You can now do things like resize the virtual monitor.

    Be aware you may need to reinstall the Guest Additions each time you update the kernel for this class.

Many of the commands that you will be running on the Linux installation within the VM will require root privileges. There are a variety of methods to elevate your user privileges on Linux. You can use any of the following methods to do so:

sudo -s
  (enter your user password when prompted)
  (perform any commands to execute as root)
exit

OR

sudo sh
  (enter your user password when prompted)
  (perform any commands to execute as root)
exit

OR

sudo (command to execute as root)

Obtaining Linux Kernel Source Code

Ubuntu 14.04 ships with the 3.19 Linux kernel. For this assignment, you will upgrade the kernel to version 4.1. Follow these steps to obtain the kernel sources:

1. Install the required software packages used for kernel development, with the folling command in the Terminal:

   sudo apt-get install g++ libqt4-dev libncurses5-dev emacs vim git indent

2. Obtain the Linux kernel sources and unpack them into the appropriate directory.
   1. Run the following commands:

      git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git linux

      This last command fetches the Linux source code repository, via git, and writes the files into the linux directory. If you are not familiar with the git version control system, you should work through the Interactive Git Tutorial before proceeding.
   2. Switch to the kernel 4.1 branch by running the following commands:

      cd linux
      git checkout linux-4.1.y

      For the rest of this assignment, the sources in the linux directory will be referred to as your working copy of the kernel, or alternatively, Linux kernel repository. If you ever need the original unmodified code, re-run the clone command in a different directory; advanced git users may also make branches within the repository.
3. Later assignments require custom kernel code. Follow these instructions to apply those changes to your copy of the kernel.
   1. Within your working copy of the kernel, run the following commands to obtain kernel patches:

      wget http://www.csee.umbc.edu/~jtang/cs421.f15/homework/0001-x86-irq-Allow-for-software-triggered-IRQ.patch
      wget http://www.csee.umbc.edu/~jtang/cs421.f15/homework/0002-x86-Create-kernel-defconfig-for-CS421-VirtualBox-VMs.patch

   2. You next need to configure git to allow patching. Run the following, substituting your information:

      git config --global user.email "gburdell@umbc.edu"
      git config --global user.name "George Burdell"

      (Note the above is dash, dash, "global".)
   3. Apply those patches using git:

      git am *.patch

Customizing Your Kernel

For this assignment, you will perform a simple modification of the official Linux kernel. You will personalize the version string with your UMBC GL username. If you have followed all of the instructions up to this point, you should have no trouble doing this. You will know if your change is correct by running the uname -a command. When you run uname -a using the stock Ubuntu kernel, note how its version string states Ubuntu is the creator.

1. Change the version string of your working copy of the kernel.
   • As an example, if I were doing the homework, this would be what should be printed out by uname -a: ... 4.1.4jtang+ #1 SMP ...
   • Hint 1: You will have to change at most one file that is distributed with the kernel — we want you to make sure you can create/apply a patch.
   • Hint 2: The file you must change is in the top directory of the kernel source tree.
   • In order to accomplish this assignment's goals, please make sure that you have changed exactly one file that is included in the kernel's source repository. There is an option to change the kernel's name in the make xconfig step below. DO NOT USE IT. You must change at least one file that is included in the Linux kernel source distribution, and you should make your change BEFORE running any of the commands in the next step.
2. Configure and compile the custom kernel and its modules.Use the CS421-supplied kernel configuration as a basis of your configuration. Run the following commands:

   make mrproper
   make cs421_defconfig

3. Now compile and install the kernel. This step will likely take quite a while, maybe an hour on an older computer. Run the following:

   make clean
   make -j3
   sudo make modules_install install

4. While the kernel is compiling, read the book Linux Kernel in a Nutshell.
5. Your working copy of the kernel should now be built and installed. In addition, the install process should have updated the bootloader (grub) to boot automatically into your new kernel.
6. Your Ubuntu VM now has two kernels that grub can use during booting. By default, grub will always select your personalized kernel. You will need the ability to select the original Ubuntu kernel, in case your customization failed. As root, edit /etc/default/grub. Set GRUB_HIDDEN_TIMEOUT to -1 and GRUB_HIDDEN_TIMEOUT_QUIET to false. Then run update-grub (again as root) to update grub's configuration. See the grub documentation for details.
7. Reboot your VM. At the grub menu, select the second option, and then your personalized 4.1 kernel. (If you do not see it, choose the original 3.19 Ubuntu kernel.)
8. Make sure your custom kernel boots properly in the VM. Run the following commands to check the customized kernel and prepare the first part of your homework submission:

   uname -a
   mkdir $HOME/hw1
   uname -a > $HOME/hw1/uname.out

9. Verify that the output from the last command shows the correct version string, as specified earlier. If it does not match the format shown above, try again. For obvious reasons, your date string in the kernel's uname will be different. Also, the "#1" is a counter for how many times the kernel has been compiled out of the source tree in use, so if you have to try more than once, that will be different as well. To recompile the kernel, only perform step 3 above.
10. Don't forget to reinstall VirtualBox Guest Additions after upgrading your kernel.

Creating Kernel Patches

Your next objective is to learn about patches and diffs. Since many modifications to the kernel are small (compared to the total kernel source size), such updates are usually distributed in the form of a patch file; those patch files describe differences between the original software and the modified code. If you make minor modifications to the kernel (such as for CS421 projects) you will want to create patches containing the diffs. Remember later assignments will distribute code in the form of patches.

1. Creating diffs is easy with git. A good tutorial can be found at http://www.gitguys.com/topics/git-diff/.
2. To view the diff you made so far, run the command git diff.
3. Next, you will make a git commit.
   1. Run git add filename, where filename is the name of the file you changed.
   2. Run git commit. Using the editor, add a one-line commit message, describing your change. Blank lines and lines beginning with # will be ignored. Save and exit.
4. Export your commit like so:

   git format-patch HEAD^1 -o $HOME/hw1

Applying Kernel Patches

Now that you have created a patch file, you will apply another patch to your kernel source.

1. Using wget, fetch the file http://www.csee.umbc.edu/~jtang/cs421.f15/homework/0003-babblebot-device-that-randomly-generates-babbles.patch.
2. Use git am to apply just this patch.
3. Run make xconfig. In the left panel, navigate to Device Drivers, and then to X86 Platform Specific Device Drivers. The above patch added a new so-called platform device driver to your Linux source code. (It will have a very obvious name.)
4. Quit out of the graphical configuration tool. Recompile and install the kernel as above, so as to enable the platform driver. Reboot your virtual machine.
5. Use your favorite search to read what the Linux virtual file /proc/interrupts does. Examine your /proc/interrupts after the reboot.
6. Using your favorite text editor, create the file $HOME/hw1/platform-driver.txt. In that file, give the name of the new platform driver and a brief description of what it does. Include a summary of what /proc/interrupts is for and how this platform driver affected /proc/interrupts. Update: patch 0003, when applied, does not modify /proc/interrupts, due to a piece accidentally omitted. You still need to describe what /proc/interrupts does in your homework submission.

Submission Instructions

Ssubmit the archive hw1.tar.bz2 containing the following files:

• uname.out — The output of you running the uname -a command.
• 0001-your_commit_message.patch — Your customization of the kernel.
• platform-driver.txt — Description of what the third patch did. Include link(s) to which website(s) you used to read up on /proc/interrupts.

You can create the hw1.tar.bz2 file as follows:

cd $HOME/hw1
... copy the files requested into this directory if they are not already there ...
cd $HOME
tar cjvf hw1.tar.bz2 hw1

Once you create this tarball, upload it to the UMBC gl server like so:

scp hw1.tar.bz2 glusername@linux.gl.umbc.edu:

Then run the submit command as described at the top of this page to submit your work and gain credit for this assignment.