A Quick-Fire chroot Environment
By Ben Okopnik
As a teacher in the computer field, I often get invited to teach at various companies which are not necessarily prepared for the requirements of a computer class - at least not in the finer details. In this article, I'd like to share one of the software tools I use to handle some of the unexpected (or should I say, the "expected unexpected"?) situations.
I recall visiting a well-known company, several years ago, that lost track of the time the class was supposed to start... as well as the fact that I would need to get through their security. And that the students needed the books that had been ordered for them (these books were actually in the very next room, but the people who delivered them didn't manage to route the information to the people responsible for the class.) And their computers were set up with Windows - despite the fact that the class was called "Introduction to Unix" - and the Unix servers they were supposed to connect to were not available (something about firewalls, and the only sysadmin who could configure them being out that day.) Oh, and there was no Internet access from there, either.
That, my friends, is what you might call a scorched-earth disaster area. At the very least, that's a bad, bad situation to walk into on a fine Monday morning, with a long week of teaching stretching out into forever ahead of you.
So, what happened? Well, I not only survived that one, I got excellent reviews from all the participants. Yes, I'm fast on my feet. Yes, I carry a lot of what I know in my head instead of being completely dependent on a book. Yes, I can take charge and handle crazy situations while other people are reeling and writhing and fainting in coils [1]. But one of the major factors that saved my bacon that day was that I could quickly and safely set up a bunch of login accounts - i.e., give each of my students an individual "sandbox" to play in, loaded with a bunch of basic Unix/Linux tools - without compromising my system, or letting them damage or destroy it by accident while mangling everything they could lay their hands on (as new students, or anyone experimenting without restraint, will.)
I did all that by configuring a chroot environment: i.e., one where the root ('/') of the login system resides somewhere other than my own system's root does (e.g., '/var/chroot'). Since, as you're probably aware, there's no way [2] to go above the root directory, the users are effectively "trapped" inside the created environment - and you have a safe "sandbox" that only contains the tools that you put in there. It's a very handy tool - actually, a set of tools - if you suddenly need to do this kind of thing.
Thinking about the process
In order to have a usable Linux environment, there are several basic items
that are necessary. For one, you need a minimal directory structure
(/etc for the required configuration files; /dev
for the devices; /lib and /usr/lib for all the
required libraries for the programs that expect to find them there;
/bin, /usr/bin, and /usr/sbin for
all the required programs, and so on.) For another, you need the
configuration files, the libraries, and all the external files used by all
the programs you want to run - this actually becomes quite involved,
especially if you want the man pages that go along with these programs!
Then, of course, you also need the devices, the log files, the actual
config files (/etc/passwd, /etc/shadow, and so
on)... and then you need the user accounts and any files you want to
provide for those users.
Whew. This is beginning to sound complicated. Is there an end to this, or does it just keep getting bigger and bigger?
Rejoice! Linux is at hand!
Fortunately, there is indeed an end to it - and it's not that far off. Not much more than I described above, in fact. Also, with a bit of thought (and the amazing tools provided with Linux), much of this can be automated.
Here's the script that I use to build the whole thing. It takes an optional argument specifying how many login accounts to create (it defaults to 20 if no argument is given.) I'll intersperse my comments throughout; the running version, along with the other tools that I'll describe below, is available here.
#!/bin/bash
# Created by Ben Okopnik on Thu Mar 22 22:50:21 CDT 2007
[ "$UID" -eq 0 ] || { echo "You need to run this as root."; exit 1; }
You have to be root to do this stuff, of course; many of the files and directories you need to copy are only readable by the root user.
# If a number has been specified as a command-line arg, use it; otherwise,
# create 20 accounts.
if [ -n "$1" ]
then
if ! [[ "$1" =~ ^[0-9]+$ ]] || [ "$1" -le 0 -o "$1" -gt 100 ]
then
echo "If used, the # of accounts to create must be 1-100. Exiting..."
exit 1
fi
fi
# Default to 20 accounts unless some other number has been specified
number=${1:-20}
I limited it to a max of 100 users because that's way, way more people than I'd ever teach in one class - plenty of safety margin. You're welcome to modify that number, but you should consider carefully if there will be any adverse unintended consequences from doing so.
source .chrootrc
This is where I get the variables that tell me where to set up the environment, what IP to use for accessing it, and what netmask to use with that IP. (See the "chrootrc" file in the tarball if the method is less than obvious.)
echo "Creating the basic dir structure"
mkdir -p $dir/{bin,dev,etc,lib,proc,tmp,var}
mkdir -p $dir/usr/{bin,lib,local,sbin,share}
mkdir -p $dir/usr/local/share
mkdir -p $dir/var/log
That's all the directory structure that's necessary for now. Later, we'll add more stuff as needed - see the section on program installation.
echo "Creating devices in $dir/dev" mkdir $dir/dev/pts mknod -m 666 $dir/dev/null c 1 3 mknod -m 666 $dir/dev/zero c 1 5 mknod -m 666 $dir/dev/full c 1 7 mknod -m 655 $dir/dev/urandom c 1 9 mknod -m 666 $dir/dev/ptyp0 c 2 0 mknod -m 666 $dir/dev/ptyp1 c 2 1 mknod -m 666 $dir/dev/ptyp2 c 2 2 mknod -m 666 $dir/dev/ptyp3 c 2 3 mknod -m 666 $dir/dev/ttyp0 c 3 0 mknod -m 666 $dir/dev/ttyp1 c 3 1 mknod -m 666 $dir/dev/ttyp2 c 3 2 mknod -m 666 $dir/dev/ttyp3 c 3 3 mknod -m 666 $dir/dev/tty c 5 0 mknod -m 666 $dir/dev/ptmx c 5 2
The required devices, of course...
# Create the 'lastlog' file touch $dir/var/log/lastlog
...as well as the one required logfile (if I recall correctly, you either can't log in or SSH in without it.)
echo "Copying the basic toolkit [this takes a while]"
cp -a /etc/ssh $dir/etc/ssh
# cp -a /bin/{bash,cat,chmod,cp,date,ln,ls,more,mv,rm} $dir/bin
# cp -a /usr/bin/{clear,env,groups,id,last,perl,perldoc,a2p,pod2man,cpan,splain} $dir/usr/bin
cp -a /usr/sbin/sshd $dir/usr/sbin/
cp -a /usr/lib/perl* $dir/usr/lib
cp -a /usr/lib/man-db $dir/usr/lib
cp -a /usr/share/perl* $dir/usr/share
cp -a /usr/local/share/perl* $dir/usr/local/share
ln -s $dir/bin/bash $dir/bin/sh
echo "echo chroot$dir" > $dir/bin/hostname
chmod +x $dir/bin/hostname
Here, I manually copy in both the required files and the ones that I use in my Linux intro, shell scripting, and Perl classes. This is a little crude, at least now that I have an automated procedure for copying not only the programs but all their required libs, ini files, man pages, and so on (see the discussion of this below) - but that's no big deal, since I designed this whole thing to be highly redundancy-tolerant. This section is actually a hold-over from the first few versions of the script, but it definitely does no harm, and is possibly (I haven't tested for this) a requirement for keeping the entire process going. Anyone who wants to experiment, feel free to rip out this section and give it a shot.
echo "Installing the required libs for the toolkit progs"
# Different versions of 'ldd' produce several variations in output.
# Debian's includes the paths to the libraries, so deciding what to copy
# where is easy; if yours does not, you'll have to check if it exists in
# the standard paths and copy it to the appropriate place in the chroot
# tree.
for lib in `ldd $dir/bin/* $dir/usr/bin/* $dir/usr/sbin/*|\
perl -walne'print $1 if m#(\S*/lib\S+)#'|sort -u`
do
# Extract the original lib directory
d=${lib%/*}
# Cut the leading / from the above
ld=${d#/}
# Check if the dir exists in the chroot and create one if not
[ -d "$ld" ] || mkdir -p $dir/$ld
[ -e ${lib#/} ] || cp $lib $dir/$ld
done
This was my original method for figuring out the libraries required by the copied programs: I ran "ldd" on the whole kit of them, sorted the list of libs reported, made the list unique, and copied those libs into the environment. It works well - but does not take care of the required files, external programs, and so on. Again, I've greatly improved on this process (see the program installation section, below) - but this does no harm, and, again, may be essential.
# Create nsswitch.conf for SSH...
for n in passwd group shadow networks protocols services ethers rpc
do
printf "%-15s%s\n" $n: files >> $dir/etc/nsswitch.conf
done
printf "%-15s%s\n" hosts: "files dns" >> $dir/etc/nsswitch.conf
# ...and copy the appropriate libs
cp /lib/libnss_{files,dns}.so.2 $dir/lib
One of the required config files that's referenced by much of the networking stuff - SSH, etc. - is the Name Service Switch configuration file, a.k.a. /etc/nsswitch.conf. We need it really early on - so we just copy it in right here. Ditto the libs that are required for accessing it.
echo "Creating the user files" cat <<! > $dir/etc/profile PATH=/bin:/usr/bin:/sbin:/usr/sbin PS1='\$USER@\`hostname\`:\$PWD\\$ ' export PATH PS1 umask 0022 ! cat <<! > $dir/etc/passwd root:x:0:0:Admin user:/:/bin/bash sshd:x:111:65534::/var/run/sshd:/bin/false nobody:x:65534:65534:nobody:/nonexistent:/bin/sh ! cat <<! > $dir/etc/group shadow:x:42: utmp:x:43: ssh:x:109: nogroup:x:65534: ! cat <<! > $dir/etc/shadow root:!:14384:0:99999:7::: nobody:!:13592:0:99999:7::: sshd:!:13592:0:99999:7::: !
Hopefully, all of the above are really obvious: these are the files in /etc that are required for user validation/authentication and so on. Actually, these are just the skeletons comprised of non-user account info; the users will get added in just a moment. Note that the root account (along with the other non-user accounts) is configured so that you can't log in: the slot for the password hash contains a '!', meaning it's not a valid login account. I don't recall the exact reason that I have it there in the first place - but I do seem to recall that it was useful for something. No harm in leaving it there.
# Note: users' UIDs will start at 1001; the first account manually added
# after this (using 'chroot-add-user') will have a UID of 1000; all
# subsequent UIDs will go up from the max user UID. This places the
# teacher's account near the top of the passwd file, which makes it easier
# to edit.
for n in `seq $number`
do
# Create the home directory
mkdir -p $dir/home/student$n; chown $n:$n $dir/home/student$n
# Populate the ~/.bash_profile
echo -e "PATH=/bin:/sbin:/usr/bin:/usr/sbin\nLANG=C\nLC_ALL=C\nPAGER=/usr/bin/less\nexport PATH LANG LC_ALL PAGER" > $dir/home/student$n/.bash_profile
# Add reasonable entries to /etc/{passwd,group,shadow}
N=$((1000 + $n))
echo "student$n:x:$N:$N:Student account:/home/student$n:/bin/bash" >> $dir/etc/passwd
echo "student$n:x:$N:" >> $dir/etc/group
# Obvious password for students
echo "student$n:\$6\$dZ3bUzrS\$dltz7ogy5QMS4glTilclPGBh7ots09Sgs5KroTa0EucLnhsmSIHBJU5coCnUw1hJI2WQlhc7/kaIGXJH90i3m0:14384:0:99999:7:::" >> $dir/etc/shadow
done
chmod 600 $dir/etc/shadow
This is the fun part: we're actually pretty much done with setting up the chroot stuff, all except for the SSH server - so it's time to create the user accounts. For each of them, we'll create the home directory (named 'student$n', where $n is an incrementing number), set the ownership/group of that directory to that user, create a minimal .bash_profile for that user (note that the LC_ALL setting is required to prevent Perl from throwing errors telling you that your system's language needs to be configured), and create the required entries in /etc/passwd and /etc/shadow.
Crude Hack of The Day: I created a 'student' account on my system, used 'student' as a password, and copied the resulting hash from /etc/shadow into this script. It works fine, of course, but that's like using an elephant gun to kill a fly... (I removed the account immediately afterwards, so if you're a future student of mine, don't even bother trying. :)
Later, of course, I used Perl's "crypt" function to generate the salts and the password hashes for additional users (see the "chroot-add-user" script in the kit for the exact technique); this works fine if you want to add more users after you create this whole thing.
echo "Configuring the SSH environment" # Change the IP, the PAM, and the PrivSep notes in etc/ssh/sshd_config sed -i -e "s/#ListenAddress 0.0.0.0/ListenAddress $ip/" -e "s/UsePAM yes/UsePAM no/" \ -e "s/UsePrivilegeSeparation yes/UsePrivilegeSeparation no/" $dir/etc/ssh/sshd_config
SSH configuration: pretty simple, once you've figured it out. All the /etc/ssh stuff has already been copied over; now we'll just modify the relevant bits so that we can log into the chroot instead of the "parent" system.
ifconfig eth0:chroot $ip down
ifconfig eth0:chroot $ip netmask $netmask
[ -z "`mount | grep $dir`" ] && {
mount -tproc proc $dir/proc
mount -tdevpts devpts $dir/dev/pts
}
Last of all, we mount the "/proc" file system in the chroot - a necessary
bit of magic, and one you must remember to undo (via "chroot-stop") if you
decide to delete the environment - you won't be able to delete it
otherwise, even as root! We also set up the networking interface - really,
an alias to one. One of the nifty things that Linux allows you to do is to
use a single piece of hardware several times... or at least to make it look
like that. In this case, I use my Ethernet interface, eth0, to
connect to the LAN - but I also use an alias, eth0:chroot, to
provide an "attachment point" for another IP address, that of the chroot.
Am I making myself clear? No? Then read your "man ifconfig" - the part
about aliases. It only mentions numerical aliases, e.g. eth0:0, but
"ifconfig" takes named aliases as well. Quite the cool feature.
Now what?
Ummm... you know how I said we were done? Well, I lied. I mean, we are kinda done - that is, you can actually log in as "student1@your_chosen_ip" - but what do you do then?
So, we need to copy in some programs. And some man pages. And their supporting files. (If you ever want to be driven out of your mind very rapidly, try tracing down everything required by "man". I'll visit you at the asylum if you send me an email.) After I struggled with this for a while - a dozen times over several years, whenever I thought of it - I finally hit upon an idea that impressed even a cynic like me: run the program, use "strace" to list all the files it opens in the process, and copy all those files! Implementing this took a bit of script hackery and juggling of "strace" options, but the end result is just fantastic: I've never had to trace down a single missing file since. All that's required after that is copying the man pages - and that's a breeze by comparison. Oh, yes: the one requirement for the above is running the program to be installed in such a way that it will exit (for example, you can't just install "vim"; you have to use "vim -h", or "vim --version".) As you can see, this is not much of a hardship.
The technique goes like this (this section is from the "chroot-instprog" script):
for file in $(strace -s 1024 -eopen $full $args 2>&1|awk -F'"' '/[0-9]$/{print $2}')
do
# Skip device files, security-sensitive system files, and users' private files
if [ -n "`echo $file|egrep '/proc/|/dev/|/etc/passwd|/etc/shadow|/etc/group|~/home'`" ]
then
echo "Skipping $file"
continue
fi
echo "Copying $file to $dir$file"
# Create the target dir if it doesn't exist
targetdir=${file%/*}
[ -d "$dir$targetdir" ] || mkdir -p "$dir$targetdir"
if [ -f "$dir$file" ]
then
echo "$file already present; skipping..."
continue
else
# Copy, and: preserve links/mode/ownership/timestamps; update only;
# force if necessary; follow symlinks in source.
cp -aufL "$file" "$dir$file"
echo $file
fi
done
I look for successful file "open"s, then copy those files - assuming they don't already exist - creating any containing directories as necessary.
Last, we'll install the man pages. A little tricky stuff there due to the variety of ways in which they're named (and linked), but nothing a little scripting can't solve. From the same file as above:
# Find man pages, install them if they exist
# Strip out any extensions
b=${prog%%.*}
# Strip all the path info
bare=${b##*/}
m=`man -w $bare`
if [ $? -eq 0 ]
then
# Look for a manpage for the progname without any extensions
mb=${m%%.*}
# Strip off the path info
mbare=${mb##*/}
# Get the path for the manpage installation
mpath=${m%/*}
# ...and get the extension as well, since we'll need to add it back on
mext=${m#*.}
# If the manpage filename doesn't equal the progname (e.g. 'awk' and
# 'gawk'), then install the 'linked' manpage by following the link and
# saving the endpoint under the original name. Weird solution - but it
# works well as long as we don't need to update or relink the manpages
# (as we would in a real distro.)
if [ "$bare" != "$mbare" ]
then
[ -d "$dir/$mpath" ] || mkdir -p "$dir/$mpath"
echo "Copying $mpath/$bare.$mext to $dir/$mpath/$bare.$mext"
cp -fL "$mpath/$bare.$mext" "$dir/$mpath/$bare.$mext"
else
tdir=${m%/*}
[ -d "$dir$tdir" ] || mkdir -p "$dir$tdir"
echo "Copying $m to $dir/$m"
cp -fL "$m" "$dir/$m"
fi
fi
echo "Copying $full to $dir$full"
cp -fL $full $dir$full
So, how do I use this thing already?
Simple: first, reconfigure the "chrootrc" file for the location, IP, and netmask you want to use. Next, run the "chroot-create" program as root, optionally supplying an argument if you want something other than 20 accounts. If you're done at that point, or if you just want to check to see if it all worked, run "chroot-start" as root, and log into one of the created accounts (e.g., "student1" with a password of "student"). That's it.
If you want to do more than that, though, just run "list-install"; it installs a whole raft of programs, via "chroot-instprog", that I like to use in my classes (or set up your own list; the syntax of "list-install" is an obvious one.) If you want to add another user - e.g., I like to add one named "ben" so I don't have to think about which student account I should use - there's "chroot-add-user". If you need to shut down the chroot - e.g., you're tired of serving the world, and just want to run SSH so you can log into your own system - then there's "chroot-stop". That's it - and that's all I've found necessary for managing this system in practical use.
Summary
This system seems to hang together, work well, and has served me on many occasions when nothing else would do. If you find yourself in similar need, or can envision a situation in which you would, I suggest giving this kit of programs a try - perhaps getting a little practice with it. Then, when you need it, it'll go smoothly.
If you do play around with it, please feel free to contact me and let me know if you've found a bug or managed to break it somehow. I'd love to hear from you [3] - I'm always glad to improve my programs! - and detailed problem reports, suggestions for any additional features (although I can't envision many), or tales of your own successes are always welcome!
[1] Graduates of The Mock Turtle Academy abound, alas. If you have no idea of what I'm talking about, read your Lewis Caroll; as the "fortune" program says, "The best book on programming for the layman is Alice in Wonderland; but that's because it's the best book on anything for the layman."
[2] That's not precisely true; there are indeed ways to break out of a chroot environment. However, as far as I'm aware, those methods require a) root-level access within that environment, b) the awareness that it can be done, and c) a bit of savvy about such things. The first is something that I definitely do not make available (no root login, no 'sudo', 'super', or anything else of the sort); the second and the third are not something that I'd expect of brand-new students in a high-pressure learning environment. And I have a few traps set all over the place just in case, anyway. :)
[3] One of my earlier scripts, "domain-check" (a domain expiration checker that I created a while back, inspired by Ryan Matteson's earlier script and Rick Moen's article about domain expiration) has proven to be very popular, and a number of people have sent me reports about domains that weren't being registered by it (since fixed, or listed as non-reporting.) One of the reporters, however, has been tireless and amazing: a fellow named Sukbum Hong has been responsible for a good dozen changes in the script, particularly for domains in East Asia. He's got lots of credit in the script comments, and I wish there were a lot more people like him around!
Talkback: Discuss this article with The Answer Gang
Ben is the Editor-in-Chief for Linux Gazette and a member of The Answer Gang.
Ben was born in Moscow, Russia in 1962. He became interested in electricity at the tender age of six, promptly demonstrated it by sticking a fork into a socket and starting a fire, and has been falling down technological mineshafts ever since. He has been working with computers since the Elder Days, when they had to be built by soldering parts onto printed circuit boards and programs had to fit into 4k of memory (the recurring nightmares have almost faded, actually.)
His subsequent experiences include creating software in more than two dozen languages, network and database maintenance during the approach of a hurricane, writing articles for publications ranging from sailing magazines to technological journals, and teaching on a variety of topics ranging from Soviet weaponry and IBM hardware repair to Solaris and Linux administration, engineering, and programming. He also has the distinction of setting up the first Linux-based public access network in St. Georges, Bermuda as well as one of the first large-scale Linux-based mail servers in St. Thomas, USVI.
After a seven-year Atlantic/Caribbean cruise under sail and passages up and down the East coast of the US, he is currently anchored in northern Florida. His consulting business presents him with a variety of challenges such as teaching professional advancement courses for Sun Microsystems and providing Open Source solutions for local companies.
His current set of hobbies includes flying, yoga, martial arts,
motorcycles, writing, Roman history, and mangling playing
with his Ubuntu-based home network, in which he is ably assisted by his wife and son;
his Palm Pilot is crammed full of alarms, many of which contain exclamation
points.
He has been working with Linux since 1997, and credits it with his complete loss of interest in waging nuclear warfare on parts of the Pacific Northwest.
