For a very long time I’ve been a Slackware user. I still happily run Slackware on my servers, but I moved to Linux Mint for all my workstations.
One of the things that always pissed me off about non-Slackware distributions is the amount of time it takes to install new software. Sure in Slackware I had to download the source code and build it myself, but I did that once, and then literally never had to download it again for years.
On any apt-based or yum-based or similar system it seems that any time you do anything with the package manager you have to wait minutes, and sometimes hours for something to be downloaded from a mirror. And if you have more than one machine – you have to double, triple, quadruple, etc. that time.
Finally I had enough and I set up my own mirror. This post is a walkthrough of what I did.
I considered the following options:
- apt-mirror is the most commonly recommended tool. It didn’t work for me for two reasons: it’s officially no longer maintained, and even though it’s just a Perl script – it needs to run on a Debian (or derivative) server. My server is Slackware.
- ftpsync seemed overkill for my needs. And I don’t think it’s intended for partial mirrors.
- apt-cache and HttpReplicator seemed too limited for my needs.
- Rsyncmirror might be a good option but I didn’t try it.
I settled on DebMirror. It does not have any dependencies (except one Perl module), it’s very configurable, and at the same time it’s simple.
Mostly I followed that tutorial on the Ubuntu help page (which is very good) but I had to customise it to work in my environment and make it work for Linux Mint.
Step 1 – Disk space
I knew it would take a lot of disk space, but still I underestimated it. I ended up adding a cheap rotating drive to my server just to hold the mirrored files. I figured in the long term I’ll have mirrors for different distributions so the extra space won’t hurt.
Just to give yourself a clear idea: have a look at the Debian mirror size page. Even if you choose one or two architectures – you probably don’t have that much disk space sitting there unused. So figure that out before you start.
The minimum today for the Linux Mint mirror is just over 100GB. But Who knows what it will be by the time you read this.
Step 2 – Install debmirror
If you’re running an apt-based distro on your server – you use apt to install debmirror. For everyone else – you’ll have to download the source package, get the script out of there, and put it into /usr/local/bin. For me the steps looked like this (I did everything as root):
cd /root wget http://deb.debian.org/debian/pool/main/d/debmirror/debmirror_2.32.tar.xz tar xvf debmirror_2.32.tar.xz cd debmirror cp debmirror /usr/local/bin
Finding the URL to download the latest debmirror package was a pain, but I don’t think the version matters that much.
At this point on my server when I ran it, I got an error like this:
# debmirror Can't locate LockFile/Simple.pm in @INC (you may need to install the LockFile::Simple module) (@INC contains: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at ./debmirror line 592. BEGIN failed--compilation aborted at ./debmirror line 592.
I already used CPAN before, so the fix was easy:
cpan install LockFile::Simple
Then it runs, but doesn’t do anything yet:
# debmirror mirrordir not specified Usage: ./debmirror [options]
Later I found that the version of gpgv On Slackware 14 doesn’t understand the –output argument which the debmirror script uses, so I had to edit debmirror and remove the “–output –” from this line:
"my @gpgv = qw(gpgv --output - --status-fd);"
I don’t know what that was intended to do, but it worked fine for me.
Step 3: Signatures
The Ubuntu guide didn’t explain what this does, and I didn’t bother figuring it out. Probably debmirror will verify the integrity of the mirrored packages.
I needed both the Ubuntu and Mint keys, and since the server wasn’t running either – I had to scp them from one of my workstations (10.0.0.150):
mkdir /root/mirrorkeyring
cd /root/mirrorkeyring
scp andrew@10.0.0.150:/usr/share/keyrings/ubuntu-archive-keyring.gpg .
gpg --no-default-keyring --keyring /root/mirrorkeyring/trustedkeys.gpg --import ubuntu-archive-keyring.gpg
scp andrew@10.0.0.150:/usr/share/keyrings/linuxmint-keyring.gpg .
gpg --no-default-keyring --keyring /root/mirrorkeyring/trustedkeys.gpg --import linuxmint-keyring.gpg
You’ll use this /root/mirrorkeyring directory in your scripts.
Step 4: Create mirror scripts
Linux Mint is mostly Ubuntu. But it has a couple of gigabytes of its own packages. So you’ll need to make a script to mirror the Ubuntu repositories and another (almost identical) script to mirror the Mint repository.
The comments in the script in the Ubuntu help page are quite useful, but I removed them here for brevity.
# cat /root/mirror-ubuntu.sh
#!/bin/bash
export GNUPGHOME=/root/mirrorkeyring
arch=i386,amd64
section=main,restricted,universe,multiverse
release=jammy,jammy-updates,jammy-backports,jammy-security
server=mirror.csclub.uwaterloo.ca
inPath=/ubuntu
proto=http
outPath=/srv/httpd2/htdocs/mirror/ubuntu
debmirror -a $arch \
--no-source \
-s $section \
-h $server \
-d $release \
-r $inPath \
--progress \
--method=$proto \
$outPath
Here are my notes for these variables:
- arch needs both i386 and amd64. I don’t know why. My workstations are amd64 but apt complained that the server was missing the i386 Packages files.
- section and release I figured out by looking at the /etc/apt/sources.list.d/official-package-repositories.list file on a Linux Mint box of the correct version (Linux Mint 20 in this case). I expect that in the future only the “focal” part will ever need to be changed.
- server and inPath were also from that original sources list file, I always used the mirror.csclub.uwaterloo.ca in the past (thnks guys!).
- outPath is where the mirrored files will be stored on your server. In my case that’s just a symlink to where I mounted my extra hard drive.
Then I made a copy of that script to handle mirroring Mint instead. I didn’t see any immediately obvious way to do both from one script:
# cat ~/mirror-mint.sh
#!/bin/bash
export GNUPGHOME=/root/mirrorkeyring
arch=amd64,i386
section=main,upstream,import,backport
# Mint versions:
# 20 ulyana
# 20.1 ulyssa
# 20.2 uma
# 20.3 una
# 21 vanessa
# 21.1 vera
# 21.2 victoria
# 21.3 virginia
release=virginia
server=mirror.csclub.uwaterloo.ca
inPath=/linuxmint-packages
proto=http
outPath=/srv/httpd2/htdocs/mirror/linuxmint-packages/
debmirror -a $arch \
--no-source \
-s $section \
-h $server \
-d $release \
-r $inPath \
--progress \
--method=$proto \
$outPath
It’s almost identical. The only that differ are the section, release, and the two paths.
Step 5: First sync
The first time you run the mirror-ubuntu.sh script it’s going to take a very long time. For me on a 25Mbps connection it took about 24-36 hours.
Thankfully because you’re doing it properly – this can be interrupted at any time and it will resume from where you left off with very little overhead.
When it’s almost done you should be able to see something like this:
# cd /srv/httpd2/htdocs/mirror # ls linuxmint-packages/ lost+found/ ubuntu/ # ls linuxmint-packages/ dists/ pool/ project/ # ls ubuntu/ Archive-Update-in-Progress-littlesvr.ca dists/ pool/ project/ # ls ubuntu/pool/ main/ multiverse/ restricted/ universe/ # ls ubuntu/pool/main/ a/ d/ g/ j/ liba/ libd/ libg/ libj/ libm/ libp/ libs/ libv/ liby/ n/ q/ t/ w/ z/ b/ e/ h/ k/ libb/ libe/ libh/ libk/ libn/ libq/ libt/ libw/ libz/ o/ r/ u/ x/ c/ f/ i/ l/ libc/ libf/ libi/ libl/ libo/ libr/ libu/ libx/ m/ p/ s/ v/ y/ # ls ubuntu/pool/main/a/aalib/ libaa1-dbg_1.4p5-46_amd64.deb libaa1-dev_1.4p5-46_amd64.deb libaa1_1.4p5-46_amd64.deb libaa1-dbg_1.4p5-46_i386.deb libaa1-dev_1.4p5-46_i386.deb libaa1_1.4p5-46_i386.deb # du -hs ubuntu/ 100G ubuntu/ # du -hs linuxmint-packages/ 1.7G linuxmint-packages/
Step 6: Web server setup
I didn’t need to do anything here, /srv/httpd2/htdocs/mirror was already served automatically by my Apache.
Step 7: Client setup
For some reason there doesn’t seem to be a user-friendly way to add a custom mirror to a Debian-based distribution. You have to edit text files. Oh well, on my workstations I edited /etc/apt/sources.list.d/official-package-repositories.list like this:
deb http://10.0.0.2/mirror/linuxmint-packages virginia main upstream import backport deb http://10.0.0.2/mirror/ubuntu jammy main restricted universe multiverse deb http://10.0.0.2/mirror/ubuntu jammy-updates main restricted universe multiverse deb http://10.0.0.2/mirror/ubuntu jammy-backports main restricted universe multiverse deb http://10.0.0.2/mirror/ubuntu/ jammy-security main restricted universe multiverse #deb https://mirror.csclub.uwaterloo.ca/linuxmint-packages virginia main upstream import backport #deb http://mirror.csclub.uwaterloo.ca/ubuntu jammy main restricted universe multiverse #deb http://mirror.csclub.uwaterloo.ca/ubuntu jammy-updates main restricted universe multiverse #deb http://mirror.csclub.uwaterloo.ca/ubuntu jammy-backports main restricted universe multiverse #deb http://security.ubuntu.com/ubuntu/ jammy-security main restricted universe multiverse
My entries are at the top, the original ones commented out at the bottom. Basically I just changed the server URL, leaving everything else as it was.
Note that I only have one server, that hosts everything I need from packages.linuxmint.com, archive.ubuntu.com, and security.ubuntu.com
Step 8: Try it
Run and apt-get update and apt-get upgrade on the client, and be amazed at how all the donwloading from the servers takes no time whatsoever!
If only I knew I was going to use Linux Mint for so long – I would have set this up years ago.
I’ll probably do the same eventually for CentOS which I use for work, and the waiting for downloads is equally frustrating.