Joe Desbonnet, firstname.lastname@example.org
Software version: 0.2.0, 28 Feb 2006
Not long after a new Fedora Core release, the bandwidth required to update a fresh installation can become burdensome for low/medium bandwidth internet connections. Some update RPMs can be over 80 MBytes in size, yet the update may be almost identical to the original RPM on the distribution disk.This software presents two solutions to administrators of Fedora computers:
In addition to the virtual update repository, this software distribution also contains the tools needed to create and maintain a delta repository.
The most significant change since the last release is that all my own binary difference code has been removed. Instead I rely exclusively on the SUSE DeltaRPM tool by Michael Schroeder.
See release notes and ChangeLog for details on recent changes
This software is released under the terms of the GPL v2 license. See LICENSE.txt in the distribution for details.
This software has four major dependencies:
DeltaRPM is available from this FTP location ftp://ftp.suse.com/pub/projects/deltarpm/. Download and follow the make/install procedure (ie just 'make install'). This software has been tested with version 3.3 of DeltaRPM. This software will look for the DeltaRPM binaries in /bin, /usr/bin and /usr/local/bin.
Copy RPMDC.war into the Tomcat webapps directory. You may need to restart the Tomcat server.
The following instructions assume that your Tomcat server is running on http://localhost:8080/. Substitute with the appropriate host name and port number if this assumption is incorrect.
Open a browser on the home page: http://localhost:8080/RPMDC/. Click on server administration and set:
You can now test the operation of the server by going back to the home page and clicking on http://localhost:8080/RPMDC/repo/.
Finally edit the yum configuration on the computers that need to be updated. On Fedora Core 4, edit Edit /etc/yum.repos.d/fedora-updates.repo. In the 'updates-released' section of the file change the baseurl to
baseurl=http://localhost:8080/RPMDC/repo/pub/fedora/linux/core/$releasever/$basearch/os/(ie replace 'download.fedora.redhat.com' with 'localhost:8080/RPMDC/repo').
Creating a delta repository is not required for the operation of the virtual repository. This tool is bundled in the same distribution for convenience.
To create a delta repository you need the following:
The creation and maintenance of the delta repository is currently accomplished with a command line tool.
Copy repotool.jar to a convenient location. Set this directory to be your current working directory. Use the following command line:
java -jar repotool.jar orig-dist-dir updates-dir delta-dir
Optional switches are:
|--version||Display software version and exit.|
|--help||Display short usage help message and exit.|
|--temp-dir=dir||A directory to use for temporary files. There must be enough space to expand a full RPM here. For Fedora Core 3 this can take over 600MBytes for OpenOffice RPMs. The temporary directory defaults to /var/tmp.|
|--no-clobber||If the delta file already exists do not regenerate it.|
|--test-delta||Test delta to ensure that recreated RPM is identical to update RPM|
|--latest-update-only||If there is more than one update for a package, generate deltas for the latest update only.|
|--packages=[pkg1[,pkg2[,pkg3...]]]||By default all packages in the updates directory are considered. This option allows only certain packages to be processed. Package names exclude version or release number (eg 'kernel', 'glibc-common').|
|--ignore-packages=[pkg1[,pkg2[,pkg3...]]]||If set, any packages listed here will be skipped.|
|--max-rpm-size=n||Do not process if source or target RPM exceeds this limit in bytes. This option is useful on computers with low memory. It is recommend that limit should be no greater than half the computer's RAM.|
|--min-saving=n||Minimum saving in bytes that must be realized by using a delta vs downloading the update RPM. If the saving is lower than this threshold then the delta is discarded. Defaults to 20KBytes.|
java -jar repotool.jar ./fc4dist ./fc4updates ./fc4deltas --latest-updates-only --no-clobber --ignore-packages=Omni,Omni-foomatic
Note: by default the repository builder will not save any deltas with compression of less than 20KBytes. This threshold can be changed with --min-saving=n (n in bytes).
There are two ways to recreate a update RPM repository from the deltas:
wget --mirror http://localhost:8080/RPMDC/repo
wget --mirror http://rpmdelta.wombat.ie/deltarepo
Now create a new directory to hold your recreated update RPMs. You can now create an update RPM repository with this line:
java -jar repotool.jar --deltas-to-updates distDir recreatedUpdatesDir deltasDirYou can verify the integrety of the resulting RPMS with rpm --checksig *.rpm
The target RPM is defined as the update RPM for which a request is made. Example mypackage-1.0-3.i386.rpm. The source RPM is defined as an RPM to which a delta is applied (usually an RPM in the original distribution, eg mypackage-1.0-1.i386.rpm).
For each target RPM a directory is created. The directory file name is the name of the target RPM with the ".rpm" suffix removed.
The target directory can hold zero or more deltas which can be applied to various source RPMs. A file called deltas.xml provides metadata about each delta in the directory.
Example: mypackage-1.0-1.i386.rpm in the original distribution. Two updates are released some time later: mypackage-1.0-2.i386.rpm and mypackage-1.0-3.i386.rpm. The repository files for these updates will be as follows:
The deltas.xml file looks like this:
<?xml version="1.0"?> <delta-manifest version="0.2"> <delta file="from_mypackage-1.0-1.i386.deltarpm" size="1234" algorithm="deltarpm" source="mypackage-1.0-1.i386.rpm" target="mypackage-1.0-3.i386.rpm" target-size="456789" /> <delta file="from_mypackage-1.0-2.i386.deltarpm" size="1211" algorithm="deltarpm" source="mypackage-1.0-2.i386.rpm" target="mypackage-1.0-3.i386.rpm" target-size="456789" /> </delta-manifest>
I have made a delta repository for the Fedora Core 4 distribution at http://rpmdelta.wombat.ie/deltarepo/fc4/i386/. This is provided without any reliability guarantees. If bandwidth consumption becomes and issue I may have to switch it off. If this software proves successful I hope that the delta repositories will be mirrored along with the distribution and updates.
For testing I recommend that you build your own repository. See section on building a repository.
To test you need to make a directory with all the FC4 RPMs from the original distribution. You can make this by copying /media/cdrom/Fedora/RPMS/* from the 4 distribution CDs or by downloading all the RPM files from a distribution mirror. For example, the following 'wget' command will download all distribution RPMS from download.fedora.redhat.com.
wget --mirror ftp://download.fedora.redhat.com/pub/fedora/linux/core/4/i386/os/Fedora/RPMS/
Please use your nearest mirror site if possible.
I found the VMWare server (http://www.vmware.com/) useful for testing. This software is free of charge, but is not open source. I was able to install a fresh FC4 installation and upgrade using this system (although I did have to run yum -y update several times before it was successful -- see note in release notes).
This software is in development and should be considered 'alpha' grade software. Protocols, file formats, schemas, repository layout and repository locations may change as the software matures.
There is no time table for future releases. If it proves useful I hope to release version 1.0 a few months after the release of FC5.
Any feedback will be greatly appreciated. In particular I'm looking for feedback on the nomenclature used ('delta' vs 'diff', 'proxy server' vs 'virtual repository' etc). Also the format of the delta repository and the schema of the delta.xml manifest file.
The current version has some obvious limitations which I plan to remove in future releases: