CMSC 421: Principles of Operating Systems

Homework 1: Building a Custom Linux Kernel

This homework is due on Tuesday, February 10, at 11:59:59 PM (Eastern standard time). You must use the submit to turn in your homework like so: submit cs421_jtang hw1 hw1.tar.bz2

All of your homework assignments must run under 32-bit Linux, specifically Ubuntu 14.04 LTS ("Trusty Tahr"). All of the projects will require making changes to the Linux kernel. In addition, all projects will be performed inside of a Virtual Machine for various reasons (uniformity of results, for instance). In order to do the later assignments in this course, you must become familiar with running Linux on your own computer. For projects, you will be compiling the kernel inside the VM and submitting patches to the kernel source code. In this homework, you well become familiar with the procedures of installing and updating the kernel inside a VM.

This homework is divided into several parts that will help you perform the following objectives:

Hardware and Software Requirements

Flash Drives and USB Hard Drives

If you choose to work in the lab, you will need an external USB hard drive or flash drive for development as well as for backups. You will need to have at least 20GiB of free space on the drive to store your VM image on it, so a 32-GiB or larger drive is highly recommended. Also, you will need to format the drive as some type of filesystem that supports files of size > 4GiB. Many flash drives come pre-formatted as FAT32, which does not support files with a size >= 4GiB. FAT32 will not work for this reason! You should format your drive as either NTFS (if you will be using Windows with the drive as well) or a Linux filesystem such as ext3. In addition, some earlier versions of VirtualBox complain when the virtual machine image resides on an ext4 volume, so you may want to avoid ext4 when storing images.

Linux on VirtualBox

In order to aid development of the projects in this course, we will be running the projects under virtualization. Virtualization allows us to run a Virtual Machine on an actual physical machine. In this way, we can run a second guest operating system inside the regular host operating system. To do the projects in this class, we assume you will have access to a relatively modern PC that can run the VirtualBox VM. VirtualBox requires an x86 CPU with a decent amount of RAM. In addition, it would be advantageous if the CPU on your host machine supported the x86 Virtualization extensions (VT-x for Intel processors, or AMD-V for AMD processors). For more information about hardware and software requirements for VirtualBox, please consult the VirtualBox website. All assignments are expected to run in 32-bit mode on VirtualBox.

VirtualBox is available on the machines in the ITE 240 lab under both Linux and Windows. You can also download it for free from https://www.virtualbox.org/wiki/Downloads to run under your own Windows, Mac, or even Linux host operating system.

For the purposes of assignments in this course, we will be using the x86 (32-bit) version of the Ubuntu 14.04 Linux distribution. In addition, the custom kernels that will be built in this course will be based on the Linux kernel version 3.14.8 (the latest long-term stable kernel as of this writing).

Installing Linux

Perform the following tasks to create the environment that will be used to complete assignments in this course. These instructions are based upon a WikiHow article.

  1. Install VirtualBox on your computer.
  2. Download the Ubuntu 14.04.1 LTS x86 LiveCD. Make sure you select 32-bit from the drop-down menu. You can also choose to make a donation to Ubuntu, if you so desire.
  3. Scan through the Ubuntu release notes. Probably a lot of it will be incomprehensible. These notes will make more sense as you gain more experience in your Linux career.
  4. Create a Virtual Machine for your Ubuntu 14.04 installation.
    1. Using VirtualBox, create a new virtual machine. You must give it a name, such as 421VM. Give ample memory (at least 1 GiB) and virtual hard disk space (at least 20 GiB).
    2. Now that you have created the virtual machine, set the boot device to the Ubuntu ISO image you downloaded above. Power on the virtual machine to boot into the Ubuntu LiveCD.
    3. Run the Ubuntu installer. This will take a while, as that the installer will download additional files from the Internet.
    4. After installation, shut down your VM. You should now have an installation of Ubuntu on the VM that does not require the LiveCD image.
    5. Remove the LiveCD from the list of disks in the VM.
    6. Reboot the VM, and ensure that it boots properly.
  5. Use Firefox to ensure that you can connect to the Internet within your VM.
  6. Open a Terminal by clicking on the Ubuntu icon in the top-left corner. In the search box, type in terminal and hit enter to launch the Terminal program. Now right-click the Terminal icon and select Lock to Launcher.
  7. Update the packages installed on the VM by executing the following command in the Terminal: 
    sudo apt-get update && sudo apt-get upgrade
    You will need to enter your password to run the operation as root.
  8. After updating, reboot the VM, to ensure that all updates complete before proceeding. The kernel will be updated by apt, and you must reboot to have the kernel update applied.
  9. To make your experience with the virtual machine smoother, you should next install the VirtualBox Guest Additions. While running your Linux VM, choose Insert Guest Additions CD from the Devices menu. Run the installer and reboot for a third time. Note that you may need to reinstall the Guest Additions each time you update the kernel for this class.

Many of the commands that you will be running on the Linux installation within the VM will require root privileges. There are a variety of methods to elevate your user privileges on Linux. You can use any of the following methods to do so: 

sudo -s
  (enter your user password when prompted)
  (perform any commands to execute as root)
exit
OR 
sudo sh
  (enter your user password when prompted)
  (perform any commands to execute as root)
exit
OR 
sudo (command to execute as root)

Obtaining Linux Kernel Source Code

Ubuntu ships with the 3.13 Linux kernel. For this assignment, you will upgrade to the 3.14 kernel. The next step is to obtain the kernel sources. Follow these steps to do that:

  1. Install the required software packages used for kernel development. In the terminal, install the required packages with the following command: 
    sudo apt-get install gcc g++ libqt4-dev libncurses5-dev emacs vim perl nano patch git
  2. Obtain the Linux kernel sources and unpack them into the appropriate directory.
    1. Run the following commands: 
      cd /usr/src
sudo chmod 775 .
sudo chown root:adm .
git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git linux
      This last command downloads the latest version of the Linux source code repository, via git, and places the repository into a linux directory. If you are not familiar with the git version control system, you should work through the Interactive Git Tutorial before proceeding.
    2. Switch to the latest 3.14 version of the kernel by running the following commands: 
      cd /usr/src/linux
git checkout linux-3.14.y
      For the rest of this assignment, the sources in the directory /usr/src/linux will be referred to as your working copy of the kernel, or alternatively, Linux kernel repository. If you ever need the original unmodified code, re-run the clone command in a different directory; advanced git users may also make branches within the repository.

Customizing Your Kernel

For this assignment, you will perform a simple modification of the official Linux kernel. You will personalize the version string with your UMBC GL username. If you have followed all of the instructions up to this point, you should have no trouble doing this. You will know if your change is correct by running the uname -a command. When you run uname -a using the stock Ubuntu kernel, note how version string states Ubuntu is the creator.

  1. Read the detailed instructions for rebuilding the Linux Kernel for Ubuntu. Although the blog post refers to Ubuntu 12.04 LTS, the instructions still apply for Ubuntu 14.04 LTS.
  2. You may want to review Linux Kernel in a Nutshell as well.
  3. Change the version string of your working copy of the kernel.
    • Hint 1: You will have to change at least one file that is distributed with the kernel — we want you to make sure you can create/apply a patch.
    • Hint 2: The file you must change is in the top directory of the kernel source tree.
      As an example, if I were doing the project, this would be what should be printed out by uname -a: ... 3.14.29jtang+ #1 SMP ...
    • In order to accomplish this project's goals, please make sure that you have changed at least one file that is included in the kernel's source repository. There is an option to change the kernel's name in the make xconfig step below. DO NOT USE IT. You must change at least one file that is included in the Linux kernel source distribution, and you should make your change BEFORE running any of the commands in the next step.
  4. Configure and compile the custom kernel and its modules. This step will likely take quite a while, perhaps a few hours. Run the following commands (you will have to hit the "save" button in the xconfig step): 
    cd /usr/src/linux
make mrproper
cp /boot/config-$(uname -r) .config
make xconfig
make -j2
sudo make modules_install install
  5. Your working copy of the kernel should now be built and installed. In addition, the install process should have updated the bootloader (grub) to boot automatically into your new kernel.
  6. (Optional) Your Ubuntu VM now has two kernels that grub can use during booting. By default, grub will always select your personalized kernel. You can edit /etc/default/grub, so that at startup you can choose which kernel to boot. See the grub documentation for details.
  7. Reboot your VM, and start your customized kernel.
  8. Make sure your custom kernel boots properly in the VM. Run the following commands to check the customized kernel and prepare the first part of your project submission: 
    mkdir $HOME/hw1
uname -a > $HOME/hw1/uname.out
  9. Verify that the output from the last command shows the correct version string, as specified earlier. If it does not match the format shown above please go back and try again. For obvious reasons, your date string in the kernel's uname will be different. Also, the "#1" is a counter for how many times the kernel has been compiled out of the source tree in use, so if you have to try more than once, that will be different as well. If you do have to compile the kernel again, you do not have to run the make mrproper step again.

Creating Kernel Patches

Your next objective is to learn about patches and diffs. Since many modifications to the kernel are small (compared to the total kernel source size), such updates are usually distributed in the form of a patch file; those patch files describe differences between the original software and the modified code. If you make minor modifications to the kernel (such as for CMSC421 projects) you will want to create patches containing the diffs. Remember that you will always submit patches for every kernel project and we will apply your patches to our copy of the kernel repository when grading your project. A bad patch file may result in 0 points!

  1. Creating diffs is easy with git. A good tutorial can be found at http://www.gitguys.com/topics/git-diff/.
  2. To view the diff you made so far, run the command git diff. Save the diff into a patch file like so: 
    git diff > $HOME/hw1/customized-uname.patch

Applying Kernel Patches

Now that you have created a patch file, you will do the converse by applying a patch.

  1. From the kernel repository directory, run the following to download the CS421 patch file for this assignment: 
    wget http://www.csee.umbc.edu/~jtang/cs421.s15/homework/hw1.patch
  2. Apply the patch with the following command: 
    patch -p1 < hw1.patch
  3. Run make xconfig. Using your favorite text editor, create the file $HOME/hw1/patch.txt. In that file, describe what the effects of the patch are.

Submission Instructions

You should submit a single .tar.bz2 file hw1.tar.bz2 containing the following files:

You can create the hw1.tar.bz2 file as follows: 

cd $HOME/hw1
... copy the files requested into this directory if they are not already there ...
cd $HOME
tar cjvf hw1.tar.bz2 hw1

Once you create this tarball, upload it to the UMBC gl server like so: 

scp hw1.tar.bz2 glusername@linux.gl.umbc.edu:
Then run the submit command as described at the top of this page to submit your work and gain credit for this assignment.