I’ve been working with the various incarnations of Red Hat Linux, including Fedora, since before the company went public.  In that time I’ve done more than just a few upgrades from one release to another.  With each upgrade I learn just a little more about administering my own Linux system.   I’m not always pleased with each release but I learn to live with the quirks introduced and the ones removed.  In the end, after having used most of the other popular distributions and even in spite of my constant complaining, I’m still happy with my Fedora.

The tips and tricks to making an upgrade go smoothly are not hidden masters secrets.  They are simply the things we learn by doing.  Do them enough, you learn to do things right.  There are good ways and bad ways of doing an upgrade.  The bad ways are often the clever ways.  The good ways are usually less glamorous.  But they work.  And they work well.

Tip #1:  be mundane.  An upgrade should not be clever.

So how does one go about being mundane with an upgrade?  Follow along and I’ll tell you.

Note that the following instructions are well suited for the average user at home using Fedora.  They are not necessarily well suited to someone who administers large numbers of computers with lots of users or for other Linux distributions.  And if you follow these steps and screw up your systems, don’t blame me.  This is just how I do it and it works for me.  Caveat reader.

Step 1:  The Plan

The problem with upgrades is the incompatibilities you find between the old version and the new version.  Some application depends on some library and the new version changes the library but not the application because the application didn’t come with the distribution you’re using.  You got it later from some other repository.  This problem isn’t something to blame on the Linux distribution, it’s a systems administration problem. As it turns out, it’s usually not hard to fix for the average home user.

Tip #2:  Don’t upgrade.  Reinstall

To avoid the problem of incompatible applications and libraries, don’t do an upgrade. Wipe the old installation and install the new version in its place.  This isn’t actually hard to do and, if you plan correctly, won’t trash your personal data.  But you have to prepare yourself for this process.  The first step is to create separate partitions for systems files and your personal files.

Tip #3: Make separate partitions for / and /home.

All your personal data, including your web servers and (if possible) your databases should go under /home.  That way the distribution update will only wipe the systems files and won’t touch your personal data.  It also means you can remove any rootkits or other nasties that found their way onto your system through system files.

So the plan is:  reinstall.  But before you can do that you need to save off some important bits and pieces from the system directories.  Typically this amounts to configuration files and, as in my case, databases.

Step 2:  Backups

Before you start, you need to backup configuration files for the system tools you use on a regular basis.  This includes such things as the hosts, password and group files from under /etc, along with a slew of others depending on your situation.  Since there is no way to know exactly what files are important to you, I created a little shell script that can be edited to grab the files and directories that are important.  I call this script configfiles.sh and I run it as a cron job so that I’m always backing up those files in case of emergency.

Config files

Although I backup a number of files and directories with configfiles.sh, I find that on reinstallation I only need a few of them.  The rest I keep for reference so I can hand edit files or merge them after the reisntall completes.  Of the files I backup, the following are almost always used on the upgraded system either verbatim or merged with the new version:

• /etc/hosts
• /etc/password
• /etc/group
• /etc/resolv.conf – to avoid using my ISPs name servers I used open access, public name servers
• /etc/X11/xorg.conf – to support dual head or special configurations, depending on which machine is being upgraded.
• /etc/sane.d – Scanner configuration files
• /usr/local/ssmtp – a simplified replacement for sendmail used solely for outbound email
• /usr/local/fonts – my locally installed fonts used with the GIMP
• /etc/php.ini – I run various web servers at home that often require minor tweaks to allow large files via PHP scripts.
• /etc/my.cnf – Minor tweaks to the MySQL configuration are also required for web services or for some work related projects I do at home.
• /etc/httpd – I run multiple web servers at home, including wikis, blogs and MythWeb.
• /etc/ssh, /etc/hosts.allow, /etc/hosts.deny – My systems are protected by a firewall but they are also protected via ssh and tcp_wrapper configurations.
• /etc/wpa_supplicant – hand tweaked configurations for wireless communications with, for example, the local library.
• /etc/squid – Squid configuration files used on the server from which ssh tunnels incoming from the internet can access inside resources.

My printer configurations (/etc/cups) is also backed up but it seems I always have to redo these after each upgrade because the administrative tools for dealing with printers are evolving faster than other parts of the system and, usually, get easier to reconfigure manually with each upgrade.  Then again, I don’t do much printing at home.

Databases

The default database directories are in the system directories so that could cause some problems.  I could back up those directories but I find it easier to make dumps of the databases and then reload them after I reinstall.  The dumps are done by a the same shell script that handles all my nightly backups (see next section) which is run by cron.  This script rotates the database backups on a weekly basis.  So I always have the last 7 days of database dumps.  The dumps are stored on the /home partition so they won’t get trashed on the reinstall.

A regular backup plan

Backups should be part of any home users plans.  You can do backups to DVD or CD if you only have a single system.  Or if you’re like me and have multiple systems at home you can attach an external USB drive to one of them and backup to that system.  This is exactly what I do.  For the paranoid you can use two USB drives and duplicate the backups (I do this at work) just in case one drive fails.  I use Western Digital MyBook’s, 500GB and 1TB models, both at home and at work.  I’ve had zero failures with these and they work right out of the box with Linux (I reformat with Linux filesystems).  I’ve had terrible luck with Maxtor USB drives and would highly recommend you not use them.

My backup script is pretty much the same on every system.  I actually use one script to prep an outbound email using ssmtp that then runs the real backup script.  The backup script is configured for each system it is run on.  It requires manual updates for each system but I only need to do this once (or very seldom).  All the backups are run via rsync.  The trick with rsync, especially now that GNOME has added that nasty GVFS mount point on your users home directory, is to build proper exclude lists.

These scripts will not run on your system without modification.  You’ll need to read the scripts and update them for your personal setup.

Since the backups are stored on a remote USB drive, which can be moved to another system easily if the server it is connected to fails for some reason, I can easily restore necessary files after an reinstall.  This typically just means the database backups unless something catastrophic happens like the disk dies on the system being reinstalled.

Cron jobs

The backups are run nightly from the root user (so I can grab directories my regular user doesn’t own) using cron.  The entry looks like this:

30 1 * * * /home/user/bin/configfiles.sh /home/user/backups/hostname 2>&1 > /dev/null
30 3 * * * /home/user/bin/mail-backup.sh

The configfiles script is run first and stores the configuration files into a backup directory under my regular user.  This makes sure they get backed up when the backups are run.  The mail-backup script runs later and mails me the results of the backup process.

Step 3:  Installation

If you don’t already have your /home directories on a separate partition then backup everything under /home to an external device before proceeding.  You can create the separate partitions on this reinstallation and then copy all the files from the backup into /home.  You’ll want to backup in archive format, which means maintaining permissions, ownership and timestamps.  Check the rsync and cp commands for help on that.

Download the boot disk for the latest Fedora release and burn it to a CD.  You can copy over the DVD installation disk to another system and export it via NFS if you like, or burn it to a DVD and install from it.  You can even install from the boot disk alone via a network install across the internet, but this last option can take much longer to complete and is not something I would recommend.

Grub/Anaconda boot options

I’ve already covered the basic installation process for Fedora in my post on migrating a MythTV server to  F10.  I suggest reading that post for details on how to install Fedora if you’re doing it from a remote server with the DVD exported via NFS, as I do.  This will cover the plethora of options available to anaconda during installation.  If you install directly from the DVD, then you shouldn’t have to worry about any of that.

Partitions and filesystems

When you get to the disk partitioning section of the installation be sure to do a custom partitioning.  You only need about 20GB for the root partition (maximum, if you install absolutely everything known to Fedora and mankind).  Leave a swap space equal to twice the size of the total amount of memory you have and then the rest of the disk should be your /home partition.

Once you have the partitions set up like this then you won’t have to remake them on future reinstalls, but you’ll still do a custom partitioning so that the partitions are correctly mapped to their original mount points.

You’ll also want to make certain that the /home partition is NOT reformatted (unless this is the first time you’re creating this partition).  You should reformat the root (re: /) partition and swap space on every reinstall, however.  That’s your guarantee of cleaning out everything, plus you can change the root partition filesystem type at this point to use newer, better and faster filesystems.  At the moment I’m quite happy with ext3 but I’ve heard about some newer ones (ext4, I think, plus a few others) that may become the default in Fedora.  The safe bet it to use the default partition type on the root partition.  It’s likely to be the most stable and, as most modern file systems are journaled anyway, reliable and recoverable from the unlikey event of a system crash or power outtage.

Finish up the install and wait for the reboot.  I don’t create a user at this point because I create it manually later so it maps to my existing /home directory using the same userid and groupid.  This is easy to do from the command line but, as far as I can tell, impossible to do from the graphical install process.  So skip it when doing your reinstallation.  Just log in as root later.

Step 4:  Post Install Setup

Okay, installation has completed and you’ve rebooted and run through the First Time setup screens.  Now you’re fully rebooted and ready to use the system.  But you need to reconfigure it the way it was before.  This amounts to reconfiguring your old user (so you don’t do everything as root), configuring your network and graphics driver and copying or merging old configuration files.

Reconfigure your old user

Now it’s time to put some things back in from your previous installation, starting with your non-root user.  Look at the backup of /etc/password and /etc/group to get the information you need.  The important bits are the userid (uid), groupid (gid) and home directory.

• Group entry example (last number is the gid):  userid:x:5150:
• Password entry example (first number is the uid):  userid:x:5150:5150:Your Name:/home/userid:/bin/bash

Notice that I’ve used the same id for gid and uid.  I find it easier to track my files if I always use the same uid and gid.

Once you have the necessary information you simply create a group using the gid for your old user and a new password entry, then add your password for the new login:

groupadd -g <gid> userid

useradd -d/home/userid -c”Your name” -s/bin/bash -u <uid> -g <gid> userid

passwd userid

At this point you should do yourself a favor and add yourself to the sudo users.  As the root user just type

visudo

to edit the users file.  If you aren’t familiar with this, read the man page for more details.

Network start up and installing from backups

Now the network needs to be brought up.  Configure the network using the desktop tools – this is typically easier than trying to understand the configuration files, which for wireless configurations can get messy.  However, you can simply copy over the hosts and resolv.conf files from you backups into the /etc directory if you use static IP addresses on your networks at home.  If your system is connected directly to the internet and uses DHCP to get an IP address from your ISP then don’t copy these in from your backups – just let your ISP set them for you.

Install meld

Once the network is back up you can either mount up your remote backups or load the CD/DVD backups and copy in or merge the files from your backups.  Merging is actually pretty easy in most cases.  But you need to install meld.

This package is very useful to non-developers.  It’s a graphical program that allows you to see the differences between two files.  It’s perfect for viewing the old version of a configuration file and comparing it with the new version.  You just find your old configuration options and copy them over.  It’s pretty much point and click and works well with most simple configuration files.  The only one that took some serious though for me was squid, whose configuration file is a major pain in the butt anyway.

Installation for Fedora just requires yum:

yum install meld

You’re running as root at this point because we’re still reconfiguring and aren’t ready to login as our normal user.

Merge configurations

With meld installed you can now copy over your old configurations or merge them with the new ones.  The ones you can just copy over include:

• /etc/hosts
• /etc/resolv.conf
• /etc/hosts.allow
• /etc/hosts.deny

The format of the /etc/hosts.allow and /etc/hosts.deny files has not changed much recently so you’re probably safe copying those over verbatim, but to be safe you can always run them through meld to merge instead.  Most of the rest of the saved configuration files you should merge using meld.

In the case of httpd, you should be in the habit of configuring your local web servers as virtual hosts and placing those configurations in /etc/httpd/conf.d/custom.conf, which will allow you to just copy over that configuration without having to merge anything.

Distribution update

The next step is to make sure the installation is actually up to date with respect to security and bug fix updates from the Fedora repositories.  This is easy enough but it might take a little while to complete depending on how old your installation DVD is:

yum update

Run this and sit back for awhile.  When it’s done you can move on to the next step.

X Updates

The latest versions of the X.org server and the associated video drivers no longer require the use of an xorg.conf file.  The server is now able to figure out much of what it needs for basic operation given the chipset (aka video cards) your using.  However, to get extended features like dual head (re: multiple monitors) or to deal with some of the intricate details that have come up with the Intel drivers (which are newer and utilizing more advanced server features, and thus tend to cause more people confusion than, say, the nVidia drivers) you’ll need to configure an xorg.conf file.  Now would be the time to deal with this issue.

Personal configuration

The last step in your personal upgrade of Fedora is to set your firewall, selinux and services configurations.  These can all be done from the desktop menus.  I disable the firewalls on my system because I have a firewall running between them and the outside world that I maintain separately.  I also disable SELinux because it tends to require more administrative work than I’m willing to deal with at home.  There are a number of services that can be disabled, like ISDN.  You should see Mauriat Miranda’s excellent write ups on Fedora services for details on these.

Advanced:  User Migration

If you need to migrate multiple users you can do it just like you did with your personal user or simply write a script that merges the relevant information from /etc/password, /etc/group and /etc/shadow in your backups into your new installation.  Note that you can simply copy the relevant lines from /etc/shadow over – the passwords will simply work as they did before.  However, if you use some other password mechanism (re: you edited your PAM authentication mechanisms) then this won’t work.  Most users, however, don’t do this unless they’re connected to something like LDAP, and most home users not connected to such things.