The Answer Gang

By Jim Dennis, Karl-Heinz Herrmann, Breen, Chris, and... (meet the Gang) ... the Editors of Linux Gazette... and You!

---

Update vs Install, how best to manage /home?

From Edgar Howell

Answered By: Neil Youngman, Thomas Adam, Mike Orr, Benjamin Okopnik.

Before I go any further, here is the environment on the machine in question, SuSE 9.2 on both drives, no other OS:

/dev/hda (non-Internet drive, system doesn't even know about a modem, /etc/fstab mounts /dev/hdb2)

1  swap
2  /
3  /home

/dev/hdb (the drive booted for Internet access, /etc/fstab has no information about /dev/hda)

1  swap
2  /

a) update vs install

In part because I tend to omit a couple of releases instead of just blindly installing successive releases but also because I used to install new software into a new partition and play with it for a while before removing the previous version, in the past I have always done a clean installation. With all that entailed, creating the users again, /etc/hosts* (SOHO network) and the like.

[Thomas] When you do a new install a distro (albeit a new one, or an upgrade to a new point release of one you currently have) you can instruct it not to touch certain partition (like /home). This means you don't have to worry about loss of data. You mentioned UIDs. Backup /etc/{passwd,shadow} beforehand.

Recently I experimented with update. It worked well, avoided lots of questions and seemed really neat. But I had again skipped a couple of releases and ultimately discovered some problems.

[Thomas] I can't see where the contention is. An upgrade saves a lot of time, since all you're doing is upgrading the software, nothing more.

[Mike] I haven't used SuSE much, but that's the general problem you get when updating through multiple OS releases. Debian has special scripts you're supposed to run to switch releases; they try to automate the tricky stuff but sometimes they can't foresee what your configuration has turned into. And they have to be run in order without skipping. If you don't update packages frequently and don't follow the user forums/newsletters where potential breakages are discussed, I would stick with the clean install and copy your configuration. That way you know everything has the latest recommended configuration, whatever that is. It also provides a chance to clear out the cruft that has accumulated from packages you installed but never used; cruft that may leave droppings in the filesystem when uninstalled since it wasn't built by a clueful developer.

Alternatively, back up your entire system onto your second drive and make sure you can boot into it, then update your primary system. That way if something breaks you can just blow it away and go back to where you were.

/home isn't a big deal. If you have it on a separate partition like you do, just let the fresh install create its own home directory in the root partition. You'll have to do everything as root anyway while you're installing, so just pretend home doesn't exist. Then when everything's settled, delete the bogus home and mount your real /home partition. Same for /usr/local if you store stuff in there. I keep a /mnt/data partition with my home and local stuff, and use symlinks to get to them. That also lets me have multiple OS versions installed, all sharing the same local data. And I can unmount the local data when I'm afraid an upgrade might hurt it.

Under the old version the first user ID was 500 and under 9.2 it is 1000. That of course caused problems in the above environment: /dev/hdb under a completely new installation got new user IDs, /dev/hda under the update inherited the old ones. It was fun to re-boot into /dev/hdb after I wrote to it having booted from /dev/hda...

[Mike] The easiest way is to recreate the users with the same UIDs and GIDs they previously had. You may have to run "useradd" manually to do it. (Or "adduser" on some systems.) If your UID overlaps with an existing UID on the new system, you'll have to compromise somehow. If you give each user their own group rather than putting them all into "users", you'll have to create the group first. On my Gentoo:

groupadd -g 1001 foo
useradd -u 1001 -g foo -G wheel,floppy,audio,cdrom,dialout,cdrw foo

This is the best way if you want to boot back and forth between OS versions, you have files with unexpected owners inside the home directories, or you have programs that refer to users by UID rather than name.

Alternatively, you can just go with the new UIDs and switch the existing home directories with "chown -R USER.GROUP /home/USER". (Note that chown is going through a syntax change from "USER.GROUP" to "USER:GROUP"; you'll have to see which syntax your version supports.)

[Ben] Being the tool-using critter that I am, things like this (the word "manually", specifically) bring a shudder to the spine and a script to mind.

while read user x uid gid foo
do
	adduser --uid $uid --gid $gid $user
done < /mnt/hda<N>/etc/passwd

I suppose you could always put in a line that says

((uid+=1000))

right below the 'do'.

[Thomas] This presupposes that the users were added with "adduser" to begin with (note that UIDs from 1000+ are indicative of this). But on some systems, UIDs > 499 are used as a valid starting place for normal user IDs.

[Ben] I was walking past a haberdashery and happened to see a nice hat in the window, so I extracted a '1000' from it. };> The number would have to come from examining the password file on the current system and adapting the existing number range to what is available - obviously, there's no single 'right' answer. Sorry I didn't make that more explicit.

Oh, and the '<N>' in the '/mnt/hda<N>' isn't the explicit version either. :)

What does the Answer Gang recommend, update or clean installation?

[Thomas] An update.

[Neil] As you've noted a clean install requires you to set the whole system up again, whereas a good update should be seamless. Again as you note, there may be circumstances where the update leaves the seams showing. A clean install will normally leave you with a nice consistent system, with any cruft that was in your configuration cleaned out and everything shiny and sparkly.

Obviously, if you're changing distributions, rather than just changing versions of the same distribution then a clean install is the way to go.

Personally I incline towards doing a clean install every so often. If you're only taking every 3rd release or so, then a clean install may be worth the effort, but if you're putting every release on, then I would alternate upgrades and clean installs, or even keep the clean installs to every 3rd release.

In practice, I tend to have a number of old releases lying around in separate partitions, so I wipe an old partition and install there, then when I'm happy I've got it set up the way I like it, I copy /home across and change my default boot. This means I also have a number of old copies of my home directory left lying around.

b) managing /home etc.

I have read recommendations about distributing the various directories but assume that they only apply to environments with different physical drives (load-balancing). In this specific installation there is only one hard drive (at a time) involved.

[Thomas] This "load balancing" concept is a marketing myth, a band-wagon of terminology that's thrown around the place which people latch on to.

[Neil] Generally, I think it's about reliability more than load balancing. The point being that if some eejit fills up one partition, e.g. /home, with junk, there's still space in the other partitions, e.g. /var and /tmp, for the system to go on about it's business until the problem is rectified. If it's all in one big partition the the whole system is likely to fail.

In practice that's more applicable to big IT departments than simple home systems. At home I install everything in one big partition. It keeps things simple and I've had no problems with reliability, but I wouldn't recommend it for my work systems.

How can one best deal with update or install in order to avoid having to back up /home, waste the drive, install the software and then restore /home?

[Thomas] (see above about partitions and installation)

[Mike] For ease of use, put everything in one partition. To guard against disk I/O errors or stupid admins who don't look before they "rm -rf", put /boot, /home/ and /usr on separate partitions, make /boot and /usr readonly, and don't mount /boot by default. The reason is that if a disk sector goes bad, it may take the entire filesystem with it, but it won't disturb other partitions. Likewise, users can't mess with stuff that's mounted readonly. The down side is managing the partitions and predicting how big to make them. If one gets full but another is mostly empty, you'll have to repartition or use symlinks.

Here /home is on a different partition than the partition with the software. Will users (IDs) be created with respect to a partition /home?

[Thomas] An update doesn't concern itself with changing critical information such as that -- the only way {U,G}IDs would be affected is if shadow or login were updated -- and even then, the config files are not touched as a result.

[Mike] User IDs (UIDs) are created in /etc/passwd according to the existing entries in /etc/passwd. The smallest unused number is assigned, subject to a minimum and maximum. Where /home is mounted doesn't matter. /home doesn't have to be mounted at all if you don't use the -m option (useradd), which creates the home directory.

OK, I've been wanting to do a completely new installation on /dev/hda, let's just try it...

A little later: it turns out that things can be quite simple after all.

YaST was so unhappy at being told not to format / that I backed up the one non-system-directory on it and let YaST reformat it. Other than that and my dictating partitioning, a vanilla install, basically just accepted whatever was suggested.

[Thomas] Of course it would be unhappy about that. /etc, /lib, /bin. /sbin -- are all directories the installer will need access to. It's highly unlikely they're as their own partition (you wouldn't need nor want them to be so they're under "/".

[Mike] I've never seen a Linux installer that sniffled and sulked if it couldn't format something, but I guess there's always a first. Usually they have options to let you mount a preformatted partition.

What was really neat is that YaST had absolutely no trouble using a previously available /home! True, I had to re-create the users, but that is normal and in the process YaST notes the presence of corresponding home directories and asks whether they are to be used.

That pretty much solves that problem, for me at least. The Germans have a saying, roughly: trying something out is better than studying it.

But I'd still appreciate comments. Is there a gotcha somewhere? Hmmm, this SOHO doesn't have many users. And what about all the settings in /etc? /boot? Like would it have been possible to copy /etc/passwd and /etc/shadow from a backup? Sounds like that particular ice might be getting a bit thin...

[Neil] Settings in /etc and /boot are best created from scratch when doing a clean install IMO, especially when any of them are maintained by automated tools like YaST. There's always the possibly of significant changes between versions and just copying back your old settings can be a bit risky, although 9 times out of 10 you won't have a problem.

All IMO. Unlike many in the gang, I don't sysadmin, so others may have more authoritative answers. NOTE: We do not guarantee a consensus ;-)

[Mike] Copying the user entries from /etc/passwd is fine, as long as the numbers don't overlap. Just make sure nothing else is editing /etc/passwd simultaneously, or use "vipw" to lock it while you're editing. /etc/shadow is probably fine too, just be aware that the other distribution may have a different file location and different syntax. If it doesn't recognize the new password, you may have to restart... something.

(Actually, the UIDs can overlap if you really want two usernames treated the same. Some people use this to have a second root login with a static shell (/bin/sash). This is useful if you hose your dynamic libraries; with a static shell you can repair the damage. Just copy the root line, leave the UID 0, change the username to something else, and set the password.)

---

This page edited and maintained by the Editors of Linux Gazette
HTML script maintained by Heather Stern of Starshine Technical Services, http://www.starshine.org/

---