How Do You Select the Fastest Mirror from the Command Line?
I Want to Update My Sources. List File with the Fastest Server from the Command Line in a Fresh Ubuntu Server Install. I Know This Is Trivially Easy with the...
I want to update my sources.list file with the fastest server from the command line in a fresh Ubuntu Server install. I know this is trivially easy with the GUI, but there doesn't seem to be a simple way to do it from from the command line?
14 Answers
You don't have to do any searching anymore - as ajmitch has explained, you can use deb mirror to have the best mirror picked for you automatically.
apt-get now supports a 'mirror' method that will automatically select a good mirror based on your location. Putting:
deb mirror:// precise main restricted universe multiverse deb mirror:// precise-updates main restricted universe multiverse deb mirror:// precise-backports main restricted universe multiverse deb mirror:// precise-security main restricted universe multiverseon the top in your
/etc/apt/sources.listfile should be all that is needed to make it automatically pick a mirror for you based on your geographical location.Lucid (10.04), Maverick (10.10), Natty (11.04), and Oneiric (11.10) users can replace
precisewith the appropriate name.
Here's one way that will always work, using good old netselect and some grep magic:
The terminal-addict's "find best server" hack!
Download and
dpkg -inetselectfor your architecture from the Debian website. (it's about 125 KB, no dependencies)Find the fastest Ubuntu mirrors from your location, either up-to-date or at most six hours behind with this (I'll explain it below, sorry it doesn't split up nicely in Markdown)
sudo netselect -v -s10 -t20 `wget -q -O- | grep -P -B8 "statusUP|statusSIX" | grep -o -P "(f|ht)tp://[^\"]*"`netselect:-vmakes it a little verbose -- you want to see progress dots and messages telling you different mirrors mapping to the same IP were merged :)-sNcontrols how many mirrors you want at the end (e.g. top 10 mirrors)-tNis how long each mirror is speed-tested (default is 10; the higher the number, the longer it takes but the more reliable the results.)
This is the backquotes stuff (don't paste, just for explanation)
wget -q -O- \ | grep -P -B8 "status(UP|SIX)" \ | grep -o -P "(f|ht)tp://[^\"]*"wgetpulls the latest mirror status from .- The first
grepextracts mirrors that are up-to-date or six-hours behind, along with 8 lines of previous context which includes the actual ftp/http URLs - The second
grepextracts these ftp/http URLs
Here's a sample output from California, USA:
60 ftp://
70
77
279
294
332
364 ftp://
378
399 ftp://
455
The "ranks" are an arbitrary metric; lower is usually better.
If you're wondering why the kernel.org Sweden-EU mirror and an NZ mirror are in the top three from California, well, so am I ;-) The truth is that netselect doesn't always choose the most appropriate URL to display when multiple mirrors map to a single IP; number 3 is also known as nz.archive.ubuntu.com!
Oneliner that selects the best mirror (by download speed) based on mirrors.ubuntu.com for your ip:
curl -s | xargs -n1 -I {} sh -c 'echo `curl -r 0-102400 -s -w %{speed_download} -o /dev/null {}/ls-lR.gz` {}' |sort -g -r |head -1| awk '{ print $2 }'
Pakket netselect-apt
dapper (net): Choose the fastest Debian mirror with netselect
[universe]
0.3.ds1-5: all
hardy (net): Choose the fastest Debian mirror with netselect
[universe]
0.3.ds1-11: all
Pakket apt-spy
dapper (admin): writes a sources.list file based on bandwidth tests
[universe]
3.1-14: amd64 i386 powerpc
Not included in newer Ubuntu due to secturity issues it seems: see: Bug report
But .. I normally just use ping to find out the speed of a connection to some location. Amount of hops and latency.
Here is a Python script I wrote that finds mirrors with the lowest TCP latency.
The script also provides bandwidth and status data taken from launchpad, and will generate a new sources.list file automatically or using a mirror chosen from a list.
A usage example that lets you choose from 5 US mirrors with the lowest latency to your machine:
$ apt-select --country US -t 5 --choose
For the command line, you can use a Python tool called apt-smart
A usage example that lets you list ranked mirrors within your country (automatically detect):
$ apt-smart -l
With -l, or --list-mirrors, you will get ( example output from Travis CI U.S. server ):
---------------------------------------------------------------------------------------------------------
| Rank | Mirror URL | Available? | Updating? | Last updated | Bandwidth |
---------------------------------------------------------------------------------------------------------
| 1 | | Yes | No | Up to date | 1.73 MB/s |
| 2 | | Yes | No | Up to date | 1.68 MB/s |
| 3 | | Yes | No | Up to date | 1.4 MB/s |
| 4 | | Yes | No | Up to date | 1.35 MB/s |
| 5 | | Yes | No | Up to date | 937.62 KB/s |
...
| 75 | | Yes | No | 1 day behind | 659.67 KB/s |
| 76 | | Yes | No | 2 days behind | 351.26 KB/s |
| 77 | | Yes | No | 4 days behind | 806.81 KB/s |
| 78 | | Yes | No | 4 weeks behind | 514.31 KB/s |
| 79 | | Yes | No | 19 weeks behind | 418.94 KB/s |
| 80 | | Yes | Yes | Up to date | 446.07 KB/s |
---------------------------------------------------------------------------------------------------------
Full URLs which are too long to be shown in above table:
1:
2:
3:
5:
...
78:
80:
Of course, apt-smart can also change your sources.list if you want to:
$ apt-smart -a
With -a , or --auto-change-mirror to discover available mirrors, rank the mirrors by connection speed and update status and update /etc/apt/sources.list to use the best available mirror.
With -c , or --change-mirror MIRROR_URL to update /etc/apt/sources.list to use the given MIRROR_URL.
Compared with other tools:
apt-smartautomatically finds where you are so you don't need to specify the country when you travel abroad.apt-smartdoes real HTTP download from each mirror to get more accurate results ( bandwidth & status ) and supports HTTP proxy, rather than usingpingand relying on launchpad 's inaccurate data.apt-smartis being maintained, whereas most other tools leave issues unfix for a long time.
You can easily install apt-smart via pip, for detailed copy'n'paste install commands and usages please see Project Readme.
I developed a simple ping-based nodejs script that tests the servers listed on and returns the fastest one:
sudo npm install -g ffum
ffum
Please let me know if you find it useful or have any suggestions (=
I know this doesn't directly answer the OP's question, but there's a button in the desktop/GUI version of Ubuntu that finds the best mirror for you. It seemed to work pretty well, so I looked into it briefly, but didn't have time to follow up.
The reason I bring it up is because I think it would be pretty straight forward and usable to make it into a command line utility.
If anyone is interested, the test seems to be located in:
/usr/lib/python3/dist-packages/softwareproperties/MirrorTest.py
Again, that's about as far as I got, but I figured I'd leave this here in case anyone wanted it. I'll probably pick back up on it when I have a little more time.
Must Read
Command That Finds Fast Mirrors
On Ubuntu 18.04 I got good results by running
python /usr/lib/python3/dist-packages/softwareproperties/MirrorTest.py
That prints a list of mirrors organized by "time" (not explained), and then I used one of the mirrors it ranked highest.