Project 1

Due by 11:59 PM on March 8, 2012

Introduction/Objectives

All of the programming projects in this course will require making changes to the Linux kernel. In addition, all projects will be performed inside of a Virtual Machine for various reasons (uniformity of results, for instance). In order to do the later projects in this course, you must become familiar with the prospects of compiling the kernel inside the VM and creating update patches to submit. Every project this semester will involve these steps, and this project is no exception. Project 1 is a very simple project that is designed to familiarize you with the procedures of installing and updating the kernel inside a VM.

This project is divided into several parts that will help you perform the following objectives:

• Becoming familiar with the tools used to build a custom Linux kernel image.
• Installing Linux and building a custom kernel.
• Modifying the Linux kernel.
• Creating a Linux kernel patch file.
• Formatting and submitting projects for this course.

Hardware and Software Requirements

Hard Drives

You should have an external USB hard drive or flash drive if you wish to work in the lab, as well as for backups. You will need to have around 12GiB of free space on the drive to store your VM image on it. You will need to format the drive as some type of filesystem that supports files of size > 4GiB. Many flash drives come pre-formatted as FAT32, which does not support files with a size >= 4GiB. FAT32 will not work for this reason! You should format your drive as either NTFS (if you will be using Windows with the drive as well) or a Linux filesystem such as ext3. In addition, VirtualBox complains about potential problems using an ext4 volume, so you may want to avoid storing your VM images on an ext4 volume as well.

Even if you do not wish to work in the lab, you will still want to make regular backups of your VM images.

Linux on VirtualBox

In order to aid development of the projects in this course, we will be running the projects under virtualization. Virtualization allows us to run a Virtual Machine on an actual physical machine. In this way, we can run a second "guest" operating system inside the regular "host" operating system. To do the projects in this class, we assume you will have access to a relatively modern PC that can run the VirtualBox VM. VirtualBox requires an x86 CPU with a decent amount of RAM. In addition, it would be advantageous if the CPU on your host machine supported the x86 Virtualization extensions (VT-x (for Intel processors) or AMD-V (for AMD processors)). For more information about hardware and software requirements for VirtualBox, please consult the VirtualBox website. All projects are expected to run in 32-bit mode on VirtualBox.

VirtualBox is available on the machines in the ITE 240 lab running Linux.

For the purposes of projects in this course, we will be using the x86 (32-bit) version of the Debian 6.0 Linux distribution. In addition, the custom kernels that will be built in this course will be based on the Linux Kernel version 3.0.4.

Installing Linux

Perform the following tasks to create the environment that will be used to complete the projects in this course:

1. Install VirtualBox on your computer.
2. Create a Virtual Machine and install Debian into it.

Many of the commands that you will be running on the Linux installation within the VM will require root privileges. There are a variety of methods to elevate your user privileges on Linux. You can use the "Root Terminal" in the Applications > Accessories menu to do so, or from a regular terminal, you can use su to do so:

su
(enter your root password when prompted)
(perform any commands to execute as root)
exit

Building a Custom Linux Kernel

The next part of this project is to obtain the Linux kernel sources and build a customized kernel. Follow this set of steps to do that:

1. Install the required software packages used for kernel development.
   1. In a terminal, as root, install the required packages with the following command:

      apt-get install build-essential qt-sdk nano patch diffutils curl fakeroot kernel-package

2. Obtain the Linux Kernel sources and unpack them into the appropriate directory.
   1. In a terminal, as root, run the following commands:

      cd /usr/src
      chmod 775 .
      curl -O http://www.kernel.org/pub/linux/kernel/v3.0/linux-3.0.4.tar.bz2
      md5sum linux-3.0.4.tar.bz2

      The kernel sources should be 76,759,291 bytes long. If you are getting a file that is not that length, then the address you are using is either wrong or down at the moment.
   2. Verify that the MD5 displayed matches the following: dff86c657cabe813bda84c72bfb93ae8
   3. Unpack the kernel sources, and make a working copy to use for the modifications for this project by running the following commands in a terminal as root (in the /usr/src directory):

      tar -xjvf linux-3.0.4.tar.bz2
      cp -ra linux-3.0.4 project1
      ln -s project1 linux

   4. For the rest of this project, the sources in the directory /usr/src/project1 will be referred to as the working copy of the kernel. The sources in the directory /usr/src/linux-3.0.4 will be referred to as the vanilla copy of the kernel. Never make any modifications to the vanilla copy of the kernel!.
3. Build and install your customized kernel.
   1. Read the Kernel Rebuild Guide (mirror link). We will be using Linux 3.0.x for this project, which is handled very similarly to Linux 2.6.x.
   2. You may want to review Linux Kernel in a Nutshell as well.
   3. Change the version string of your working copy of the kernel to 3.0.4-GLUSERNAME-cs421project1 Where GLUSERNAME is your UMBC GL username. If you have followed all of the instructions up to this point, you should have no trouble doing this. You must change at least one file in the Linux source code to do this.
      As an example, if I were doing the project, this would be what should be printed out by uname -a (the hostname on my VM is yuki-vm):
      Linux yuki-vm 3.0.4-lsebald1-cs421project1 #1 SMP Wed Feb 22 16:27:55 EST 2012 i686 GNU/Linux
      Your version string must be all lowercase, otherwise you will not be able to build the kernel properly for Debian.
   4. Configure and compile the custom kernel and its modules. This step will likely take quite a while. Run the following commands in a root terminal:

      cd /usr/src/linux
      make mrproper
      make xconfig
      make KDEB_PKGVERSION=project1 deb-pkg
      dpkg -i ../linux-image-3.0.4-GLUSERNAME-cs421project1_project1_i386.deb
      

      It is highly recommended to try to cull out some unneeded device drivers from your kernel during the configuration phase. The default kernel configuration builds MANY drivers that we will not need at all. If you do not cull out some of the drivers, not only will you be wasting a lot of your own time waiting for the kernel to compile, you will need to have a lot more disk space. You can use "lsmod" inside your VM to get an idea of the modules that the VM is actually using to help you guide your search to cull out unneccessary drivers. Cleaning out unneccessary drivers and other modules is completely optional for this project, and will not affect your grade in any way.
   5. Your working copy of the kernel should now be built and installed. In addition, the install process should have updated your bootloader configuration to add the new entry to it (and made it the default kernel option too).
   6. Reboot your VM, and start your customized kernel.
   7. Make sure your custom kernel boots properly in the VM. Run the following commands to check the customized kernel and prepare the first part of your project submission:

      cd /usr/src
      mkdir project1-submission
      cd project1-submission
      script
      uname -a
      exit
      cat typescript

   8. Verify that the output from the last command shows the correct version string, as specified earlier in the project.

Creating and applying Kernel updates/patches

Your next objective is to learn using diff to generate kernel source patches (updates) and applying the patches to the original kernel. Since many modifications to the kernel are small (compared to the total kernel source size), such updates are usually distributed as "patches". If you make minor modifications to the kernel (e.g. for CMSC 421 projects) you will want to create a patch. Remember that you will always submit a patch file for every Kernel project and we will apply your patch to our copy of the vanilla kernel sources when grading your project. A bad patch may result in a grade of 0!

A good tutorial on how to create patchfiles using diff can be found at http://www.gnu.org/software/diffutils/diffutils.html. The typical call is diff [<options>] <from-file> <to-file>

You should have the vanilla kernel sources in the directory /usr/src/linux-3.0.4 as described earlier. You should have already built your first custom kernel as specified in the last part, by making appropriate changes to your working copy of the kernel sources.

1. Download the course kernel file ignorelist. This will permit you to create patches without first running make mrproper in the kernel sources directory.

   cd /usr/src/
   curl -O http://www.csee.umbc.edu/courses/undergraduate/421/Spring12/02/project1/ignorelist.txt

2. Create a patchfile for your kernel using the diff command:

   cd /usr/src/
   diff -rcP -X ignorelist.txt linux-3.0.4 linux > project1-submission/patchprj1.diff

   The file patchfile patchprj1.diff contains all the information the patch command needs to transform the vanilla kernel sources to your working copy of kernel sources.
   IMPORTANT: If the ignorelist does not work then you should execute make mrproper in the working copy of the kernel sources before creating the patchfile. If you forget to do this, you may get a very large patchfile.
3. Verify your patch using the patch command.
   You should always validate your patchfiles before submitting them. Make a new full copy of your vanilla kernel sources, apply your patch to it, and compare with the working kernel sources:

   rm -rf verify-linux
   cp -ra linux-3.0.4 verify-linux
   cd verify-linux
   patch -p1 < ../project1-submission/patchprj1.diff
   cd ..
   diff -r -X ignorelist.txt linux verify-linux

   The sources in the verify-linux directory should now be transformed into your working sources. The diff command above should not show any differences. For Project 1, if you see several Only in project1/..., or Only in linux/... this is also OK.
4. Examine your patchfile, note the format and that your changes are included in the file.
5. Note that all patches you submit must work with the "-p1" option. You can guarantee this by always using diff as shown above. Also, do not edit patch files once you have created them! This will likely break the patch command.

Submission Instructions

You will submit this project through your git repository, as with any other assignment for the course. You should create a project1 directory in the root of your git repository and place the following files in it:

• patchprj1.diff
• Your /usr/src/linux/.config file (may be renamed to config with no leading dot)
• Your /boot/grub/grub.cfg file
• The typescript file containing the output from uname -a

Be sure to commit and push the git repository once you have added the project1 directory and the required files!

What to do if you want to get a 0 on this project

Any of the following will cause a significant point loss on this project. Items marked with a * will result in a 0.

• Missing patchprj1.diff file. *
• patchprj1.diff file is in excess of 2KiB (2048 bytes).
• Any file listed in the Submission Instructions is missing.
• Patch file is not named patchprj1.diff.
• Extraneous files are included.

Please do not make us take off points for any of these things!