Roll your Desktop into a USB stick/CD

Introduction

Here is a short course (with some excursions) to help you take your existing Desktop, roll it into an ISO pipe and smoke a USB stick or CD. This is not a task for the faint-hearted - there are no automated scripts in here!

Part of the reason for this hands-on approach is that each desktop system will have its own quirks, so writing scripts which handle the various conditions which might arise is painful. Secondly, there are numerous choices possible - you must mix your own. Finally, I must confess that I do not feel energetic enough to write the scripts at this point. Instead, you have this article!

So if you have spent a lot of time fine-tuning your configuration and want to waste some more time putting it onto a (re)movable drive - read on.

Making a read-only root filesystem

Since the chosen medium for our ``live'' portable system is a CD or USB stick we do not want to write to it often. In fact, in the case of a CD-R we can only write to it once. It should be obvious, however, that a ``live'' system does need to write something if it is to be counted as among the living!

While many mechanisms have been suggested to handle this, we will (essentially) follow the system chosen by bootcd. To do this, we need to create a directory /wraith, an archive /wraith.cpio.gz and a script /etc/rcS.d/S01undead.sh.

The contents of the archive are rather system specific so you will need to choose its contents. However, if you are really, really impatient you can download the listing of the archive that I use and run the following:

 cd /
 cat wraith.lst | cpio -o -H crc | gzip -c -9 > /wraith.cpio.gz

Did you get a lot of error messages? No? Are you really sure that your system is almost identical to mine? Yes? Then you can skip the next subsection.

Choices, choices...

How does one find out what files need to be written to on a live system? One way would be to find all the files that have been written to on your current desktop. To do this first find out when the system was booted - a good measure of this is when the root filesystem was last mounted. So for example

 # Use your real root device in place of /dev/root.
 last_boot_time=$(dumpe2fs -h /dev/root | \
  sed -n -e's/Last mount time: * //p')

You may wish to use the log files or the last command instead. You only need to know the last boot time approximately; subtract a minute or so from it just to be on the safe side - unless you boot the system more often than that! Now create a file with that time stamp using the touch command:

 touch -d "$last_boot_time" /tmp/lastboot

You can now create the list of all files that were modified since that last boot (for simplicity we will only bother with the directories /etc and /var; you can add some other directories if you so desire):

 find /etc /var -newer /tmp/lastboot > /tmp/changed
 # And, just for fun...
 find /home -newer /tmp/lastboot > /tmp/home_changed

Have a look at these lists but don't delete them just yet. You should notice that there are three types of files that are written to on a running system.

Files that are automatically written to but can be empty at the start like log files or the mozilla cache directories.
Files that have information which we may want to change but rarely. Most configuration files fall into this category.
Automatically generated databases of system information. Since our system is going to be ``static'' we actually do not want the system to update these databases. Thumb-rule: If the file is large in size it probably belongs here!

We will split up our list, /tmp/changed, according to this classification: /tmp/write will consist of those files (mostly directories) that are empty at boot time but get written to as the system runs; /tmp/links will consist of the files that will be quasi-static - we will keep a static version of these files at boot time but we might want to change them on a running system. We will include the third category of files in /tmp/links as well, since we will not treat them differently - but ultimately you may want to change this.

We first create a directory to hold the files that will be modifiable at run-time - say /wraith. Mount a RAM-based file system on it by mount -t tmpfs tmpfs /wraith. Big Fat Warning: This file system is ephemeral and will be lost when you halt the system. If you wish, you can use the directory as-is (without the tmpfs mount) during this subsection, but don't forget to clean up its contents once you have created the archive as explained below.

In /wraith, we will create the top-level directories like etc, var, tmp and so on that we will want to write to. In these directories we will create the files as per the classification above. First, we'll do the writable but empty files:

 cd /
 cat /tmp/write | cpio -pdum /wraith

We expect these files to be empty at start-up so we will ``zero'' them. Do this only to the files for which you don't want to keep the contents. For now I assume these are all the files in the list /tmp/write:

 for file in $(cat /tmp/write)
 do
  if [ -f $file ]
  then
   > /wraith/$file
  fi
 done

Of course, we also need an empty tmp directory:

 mkdir /wraith/tmp
 chmod 1777 /wraith/tmp

Next, we create the links.

 cd /
 for i in $(cat /tmp/links)
 do
  dir=$(dirname $i)
  top=$(echo $dir | cut -f2 -d'/')
  rest=$(echo $dir | cut -f3- -d'/')
  name=$(basename $i)
  mkdir -p /wraith/$dir
  ln -s /wraith/$top.ro/$rest/$name /wraith/$i
 done
 # As a safety measure to ensure that all configuration files
 # have been created
 mkdir -p /wraith/etc
 cd /etc
 for i in *
 do
  ln -s /wraith/etc.ro/$i /wraith/etc/$i
 done

This is more complicated and needs further explanation. The idea is to make the ``static'' versions of the files available under the .ro top-level directories. So, for example /wraith/etc/hostname will be a link to /wraith/etc.ro/hostname.

To see this at work create etc.ro and var.ro as sub-directories in /wraith. For each of these directories (say etc) we run a pair of commands like the following. (Warning: Be careful here. If you haven't created all the links in /wraith/etc as above you may crash your running system).

 mount --bind /etc /wraith/etc.ro
 mount --bind /wraith/etc /etc

After these mounts, the file /etc/hostname is a link to the original hostname file which is now available as /wraith/etc.ro/hostname. Since the left-half of this link is on the RAM disk we can perform replacement surgery on it:

 vi /etc/hostname.new
 mv /etc/hostname.new /etc/hostname

On the other hand, if you want to change a file in a sub-directory of /etc, it's a little more complicated:

 mkdir /etc/X11.new
 ln -s /ram/etc.ro/etc/X11/* /etc/X11.new
 mv /etc/X11.new /etc/X11

After this you can edit the files in /etc/X11. Yes, this is quite twisted but (once you get the hang of it) not difficult to manage - especially since we expect that we will edit these files only rarely. An alternate approach is to create the directory tree under /etc in its entirety only leaving links to the files.

You can use the above mounts to test your choices of /tmp/links and /tmp/write, but the real test will come later. For now, undo the above mounts by a pair of commands like:

 umount /etc
 umount /wraith/etc.ro

You can also remove the .ro directories if you like. Finally, we create an archive of this directory:

 cd /wraith
 find . -xdev | cpio -o -H crc | gzip -c -9 > /wraith.cpio.gz

The cpio command will tell you how many 512-byte blocks you wrote. If the archive is really large (more than 1MB or so) then you probably need to re-do your choices.

Bringing the wraith to life

We need a mechanism to bring the /wraith directory into operation at boot time. To do this, install a script like the following one to run early at boot time. For example you could install the script as /etc/rcS.d/S01undead.sh.

 #
 
 # undead.sh Mount and load up the /wraith directory for use
 #
 # Version: 0.3  01-Feb-2005  
 #

 # If this has already been run then don't run it again.
 # We can't handle two wraiths!
 [ -f /wraith/live ] && exit 0

 # Create writable space
 mount -n -t tmpfs tmpfs /wraith


 # Create the directory structure
 cd /wraith
 gzip -dc /wraith.cpio.gz | cpio -idum
 cd /

 # Perform the cross mounts with bind
 # which is like a directory hard link.
 cd /wraith
 for i in *
 do
  mkdir $i.ro
  # We use mount with the -n
  # To avoid confusing the mtab
  mount -n --bind /$i /wraith/$i.ro
  mount -n --bind /wraith/$i /$i
 done
 cd /

touch /wraith/live

: exit

Finally, you edit /etc/fstab so that the root filesystem is mounted read-only at the next boot - just change defaults to read ro,defaults in the appropriate entry.

Don't stop just yet

Reboot and that's it! You have a read-only root system... or almost. Actually, it is likely that you will find a number of places where you didn't create the links you need or created the wrong links. Don't worry. You can modify the /wraith.cpio.gz archive to your heart's content. Make the changes you need to the ``live'' /wraith directory. Now copy all the changes from /wraith into /tmp/ghost. The command

 cd /wraith
 find . -xdev | grep -E -v '^./((live)|([^/]*\.ro))' > /tmp/list

will generate the newer list of files. You can unpack the older archive and compare its list of files with /tmp/list.

 mkdir /tmp/ghost
 cd /tmp/ghost
 zcat /wraith.cpio.gz | cpio -idum
 find . -xdev > /tmp/oldlist
 wdiff -a /tmp/list /tmp/oldlist

Using the differences you can see what files you need to create in /tmp/ghost in order to match it up with the running /wraith. You can save your changes by something like

 cd /tmp/ghost
 find . -xdev | cpio -o -H crc | gzip -c -9 > /tmp/wraith.cpio.gz
 mv /tmp/wraith.cpio.gz /wraith.cpio.gz

The changes will become automatic at the next boot. Of course, once you write the filesystem to a CD, you will have no chance to revise it again!

Undo

All this looks too complicated and life is too short? Just remove the script /etc/rcS.d/S01undead.sh, the archive /wraith.cpio.gz and the directory /wraith. You will have your system as pristine as before.

Making the initrd

We want our system to ``run anywhere'' - in particular, we should be able to mount our root file system whether it resides on a CD or USB stick (or perhaps even hard disk). If we use a CD then on most systems this will be on the device /dev/hdb or /dev/hdc. The USB stick usually shows up as /dev/sda or /dev/sdb. It should be relatively simple to just create a kernel which supports IDE CD drives and USB block devices. When we boot such a kernel (with the correct root=<device> parameter) the system will start up as expected on 90% of all systems that one is likely to encounter. If this is OK with you then you don't need an initrd so skip the rest of this section and read the HOWTO on building the kernel with IDE CD and US support - don't forget support for the ISO 9660 (CD), ext2 and vfat (Win95) file systems.

What about the remaining 10%? That will take 90% of the work as usual. One possible solution could be to build all the drivers of all possible CD drives, USB readers and the like into the kernel. Unfortunately, automatically probing for some of these devices will occasionally cause other devices to choke-up. It also seems like a bit of a waste to take up such enormous amounts of kernel memory for unused drivers. The solution provided by our intrepid kernel hackers is the modules+initrd mechanism which allows you to write a script that chooses which drivers to load depending on the devices found.

The steps

The boot loader (see the next section) will load the kernel and the initrd into memory. We will use a ``standard'' Debian kernel image which has essentially everything modularized (``essentially'' since we must have support for at least one file-system built into the kernel in order to load the init RAM disk - this could change if I understand initramfs better).

After the kernel has done its thing, it sets up the file-system with root on the initrd and executes /linuxrc but doesn't quite fully let go - when /linuxrc exits, the kernel executes /sbin/init. We follow Debian's choices when we visualize the boot process as follows:

linuxrc

This script doesn't do much since we want the kernel to let go and execute /sbin/init (still on the initrd).

init

The program /sbin/init on the initrd is a script that will run the following scripts:

loadmodules: This script loads the modules necessary to read the CD and/or USB stick.
script: This script will provide the subroutine mount_root to recognise and mount our chosen file system on /mnt.

The final steps of init are

Mount the root file system on /mnt and cd to it.
Execute pivot_root which makes the current directory the root directory for the kernel and mounts the old root directory at /initrd. After this our ``real'' root file-system is mounted as root.
Execute chroot . to change the root device of the current process so that /initrd is free to be unmounted. We must do this so that the RAM disk is free to be unmounted which frees its space for use by other processes.
Finally execute /sbin/init on the real root file system. This is the ``real'' init program which will initialise the live system.

Debian has already provided the package initrd-tools which automates the building of the initrd. This already creates the /linuxrc and /sbin/init needed for the initrd. So we only need to provide the scripts loadmodules and script.

Driver loading

Writing these scripts was one of the most complex steps for me as it deals with the aspect of Linux that I usually encounter the least - at least on a working system - booting! On the other hand, this is the job for which most installers and other forms of pre-install detection tools have been written. So we follow the ``teacher'' Tom Lehrer's dictum:

   Plagiarize,
   Let no one else's work evade your eyes,
   Remember why the good Lord made your eyes,
   So don't shade your eyes,
   But plagiarize, plagiarize, plagiarize...
   (Only be sure always to call it please, "research".)

   -- Tom Lehrer, "Lobachevsky"

There is a good IDE driver detection routine that is part of the standard Debian initrd. The Knoppix initrd gives us a safe order in which to load all the SCSI modules. The Linux-Live initrd has a list of the necessary USB modules to boot off a stick. So we put all these together to get routines which I call loadmodules_ide, loadmodules_scsi and loadmodules_usb. The loadmodules script on the initrd will then act as a dispatcher - it will choose which routine to run depending on what boot time parameters we give.

Still, we should do some work. So (plagiarising from the hotplug scripts) I also wrote a procedure loadmodules_pci that loads only those modules which correspond to devices in /sys/bus/pci/devices which match the data found in /lib/modules/kernel-version/modules.pcimap. This procedure makes use of the sysfs file system that was introduced with Linux 2.6.x but something similar may be possible using /proc/bus/pci in Linux version 2.4.x. The principle is that the kernel does provide a list of all the PCI devices that it found; for each such device it also provides some device information - the interface for this is the sysfs file system or (in 2.4.x) the proc file system. On the other hand, each module writer makes a list of all devices that the driver is known/expected to work for - the kernel build process writes these to modules.pcimap. By matching the two lists we should be able to load only those modules which have a matching device. This only works with PCI devices but most devices on PC's nowadays (including SCSI cards and the USB controller) are PCI devices.

Here is the script to loadmodules that resulted from the above deliberations. This scripts depends on a list of modules that are related to block devices.

Mounting the root file-system

The second script we will use provides the routine to mount the root device. Again the sysfs file system provided by the 2.6.x Linux kernel comes to the rescue. Under /sys/block we find a list of all the block devices on the system. If the root= option is given to the kernel we can check whether this block device is available. Otherwise we check each available block device to find evidence that it is our root file system - by checking for the existence of the archive, directory and script that we created above.

Using `mkinitrd`

The Debian initrd-tools package is a collection of scripts and so can be installed on any GNU/linux system (for example use the source package directly). The main script is mkinitrd which will create the standard Debian initrd. We will run this script and make some changes in order to create our ``special'' initrd. First off all create some directory say /tmp/mkinitrd.confdir. In this directory we will create the file exe containing the list of executables that we want in addition to the ``standard'' ones like /bin/sh - in our case we need /bin/grep. Next we create a list of all the additional files that we want to include; this is mainly the list of all modules that are in some way connected with the use of block devices; here is my list. Finally, we also need a configuration file. We are set to use mkinitrd with this directory as our configuration directory.

mkinitrd -r "" -k -d /tmp/mkinitrd.confdir -o /dev/null

This will tell you the name of the working directory which will be something like /tmp/mkinitrd.1234. Now you need to edit the /tmp/mkinitrd.1234/initrd/linuxrc.conf file to reflect the various file systems that you may use for your root file system. Finally, you copy the scripts you created above and generate the initrd with mkcramfs.

dir=/tmp/mkinitrd.1234
rm $dir/initrd/scripts/*
cp allmod.list $dir/initrd/etc
cp loadmodules $dir/initrd
cp script $dir/initrd
chmod +x $dir/initrd/loadmodules
chmod +x $dir/initrd/script
mkcramfs $dir/initrd initrd.img

If you build a kernel with ext2 filesystem support instead of cramfs, then you need to create an ext2 filesystem image based on the /tmp/mkinitrd.1234 directory instead.

Putting it all together

We now combine the ideas of the previous two sections. I assume that you have managed to make your root filesystem boot in a ``read-only'' mode and that you are currently running in that mode. I also assume that you have created an initrd that can boot on ``any'' machine.

I know that the latter requirement is hard to check given that you have access to only one machine at a time. Moreover, it is difficult to find friends who will agree if you say ``I have on this floppy an initrd and kernel that I would like to test on your system''; those few will not remain friends if your kernel+initrd manages to fry their system.

Selecting the boot loader

In order to boot off a CD or USB stick we need some software that can do that. The nominees are isolinux, loadlin and grub... and the winner, in this case, is... grub.

The main file for booting using grub is called stage2 or stage2_eltorito in the case of booting CD's. When these files are properly installed (see below how this is done for CD's), they are loaded and run by the booting machine. They look for a configuration file /boot/grub/menu.lst. We use a menu.lst file that looks like:

default 0

timeout 5

# Pretty colours color cyan/blue white/blue

title Debian GNU/Linux with myinitrd root (cd) kernel /boot/vmlinuz-2.6.8-2-686 root=auto ro quiet vga=791 initrd /boot/initrd.img boot

Other than the kernel and the initrd, we need stage2 and menu.lst in order to complete the list of steps given at the beginning.

Making the CD

First you need a ``pristine'' copy of the root file system. If you used the bind mounts procedure to make the root file-system read-only, then you can just do

 mkdir /tmp/pristine
 mount --bind / /tmp/pristine

You then make a compressed tree of this file system:

 mkzftree -x /tmp/pristine /hugeroom

where /hugeroom is some place with a lot of disk space. Remove the directories /hugeroom/lost+found and /hugeroom/boot from under this directory. Create an empty /hugeroom/boot directory to which we copy the kernel image and initrd. Into the /hugeroom/boot/grub directory goes the file stage2_eltorito along with menu.lst. These files will not be compressed.

We now create the CD image:

 mkisofs -R -J -z -hide-rr-moved -cache-inodes \
  -b boot/grub/stage2_eltorito -b boot/boot.cat \
  -boot-info-table -boot-load-size 32 \
  -no-emul-boot -o mylivecd.iso /hugeroom

Then we blank a CD (if necessary) and write our image to it. For a USB stick, we just create a partition and dump the entire image to this partition using dd. Since I do not have a system that can boot off a USB, I can only check the floppy based boot for such a system. Perhaps one of the readers can enlighten me on how this is to be handled for USB-booting BIOSes.

What else?

You'll probably want to add a writable /home directory to your system. You need to further customise wraith.cpio.gz for that. Another thing that you probably want to do is to fix the /etc/fstab file that goes onto the CD. Other config files may also need to be customised; /etc/X11/XF86config-4 comes to mind - for this to work ``anywhere'' it is best to use the vesa driver. Similarly, use dhcp to configure ethernet rather than a hard-coded IP address in /etc/network/interfaces. On most systems there is a hard disk and it is shame not to use it. You can set-up a swap partition after you boot from the CD - be careful not to trash the host machine though!

Afterword

Today one can find a number of GNU/Linux systems that work off Live CD's. There is Knoppix - and then there are its Klones. There is LNX-BBC, tomsrtbt, LTSP and even one called Puppy! There are the CD-based installers for the common distributions. But, I am still not satisfied. Each of these make choices that I am not comfortable with. They choose KDE, Gnome or fluxbox, when what I want is fvwm; or they choose xmms when what I want is alsaplayer (in daemon mode)... and so on.

What's wrong with Sunil's excellent article then? - just take a minimal Knoppix-like DSL and re-master it. I would object that Knoppix puts everything in a cloop image which makes it difficult to read the ``real'' contents of the CD on a generic system; further this also makes it difficult to master and/or re-master.

There are other approaches like that taken by Gibraltar, bootcd or dfsbuild or linux-live.

The first two keep the files in a compressed ISO file-system. That makes it usable ``anywhere''. I did try these but for one reason or another they didn't work for me. For example they required the installation of additional packages on my desktop.

Ultimately, it comes down to this: I'm a terribly nit-picky kind of person, and I have spent a lot of time fine-tuning my system and no one is allowed to dictate what packages I must install and how they must be configured.

[ I like this Kapil guy, and the way he thinks. :) -- Ben ]

I enjoy tinkering with such things, and so I must have a system that I understand fully. People also mentioned additional kernel features in late 2.4.x and early 2.6.x that simplify the building of a live CD. Finally, isn't it fun to ``roll your own''?

This document was translated from the LaTeX Source by H^EV^EA.

[BIO] Kapil Hari Paranjape has been a ``hack''-er since his punch-card days. Specifically, this means that he has never written a ``real'' program. He has merely tinkered with programs written by others. After playing with Minix in 1990-91 he thought of writing his first program---a ``genuine'' *nix kernel for the x86 class of machines. Luckily for him a certain L. Torvalds got there first---thereby saving him the trouble (once again) of actually writing code. In eternal gratitude he has spent a lot of time tinkering with and promoting Linux and GNU since those days---much to the dismay of many around him who think he should concentrate on mathematical research---which is his paying job. The interplay between actual running programs, what can be computed in principle and what can be shown to exist continues to fascinate him.

Copyright © 2005, Kapil Hari Paranjape. Released under the Open Publication license unless otherwise noted in the body of the article. Linux Gazette is not produced, sponsored, or endorsed by its prior host, SSC, Inc.

Published in Issue 113 of Linux Gazette, April 2005

<-- prev | next -->