How Do You Monitor the Progress of Dd?

dd is a wonder. It lets you duplicate a hard drive to another, completely zero a hard drive, etc. But once you launch a dd command, there's nothing to tell you of its progress. It just sits there at the cursor until the command finally finishes. So how does one monitor dd's progress?

0

20 Answers

Update 2016: If you use GNU coreutils >= 8.24 (default in Ubuntu Xenial 16.04 upwards), see method 2 below for an alternate way to display the progress.


Method 1: By using pv

Install pv and put it between input / output only dd commands.

Note: you cannot use it when you already started dd.

From the package description:

pv - Pipe Viewer - is a terminal-based tool for monitoring the progress of data through a pipeline. It can be inserted into any normal pipeline between two processes to give a visual indication of how quickly data is passing through, how long it has taken, how near to completion it is, and an estimate of how long it will be until completion.

Installation

sudo apt-get install pv

Example

dd if=/dev/urandom | pv | dd of=/dev/null

Output

1,74MB 0:00:09 [ 198kB/s] [      <=>                               ]

You could specify the approximate size with the --size if you want a time estimation.


Example Assuming a 2GB disk being copied from /dev/sdb

Command without pv would be:

sudo dd if=/dev/sdb of=DriveCopy1.dd bs=4096

Command with pv:

sudo dd if=/dev/sdb | pv -s 2G | dd of=DriveCopy1.dd bs=4096

Output:

440MB 0:00:38 [11.6MB/s] [======>                             ] 21% ETA 0:02:19

Other uses

You can of course use pv directly to pipe the output to stdout:

pv /home/user/bigfile.iso | md5sum

Output

50,2MB 0:00:06 [8,66MB/s] [=======>         ] 49% ETA 0:00:06

Note that in this case, pv recognizes the size automatically.


Method 2: New status option added to dd (GNU Coreutils 8.24+)

dd in GNU Coreutils 8.24+ (Ubuntu 16.04 and newer) got a new status option to display the progress:

Example

dd if=/dev/urandom of=/dev/null status=progress

Output

462858752 bytes (463 MB, 441 MiB) copied, 38 s, 12,2 MB/s
26

From HowTo: Monitor the progress of dd

You can monitor the progress of dd once it's running without halting it by using the kill command to send a signal to the process.

After you start dd, open another terminal and enter either:

sudo kill -USR1 $(pgrep ^dd$)

Or, if you're on BSD or OS X:

sudo kill -INFO $(pgrep ^dd$)

This will display the progress in the dd terminal window without halting the process (by printing to its stderr stream). For example:

# dd if=/dev/urandom of=rando bs=1024 count=1048576
335822+0 records in
335821+0 records out
343880704 bytes (344 MB, 328 MiB) copied, 6.85661 s, 50.2 MB/s

If you would like to get regular updates of the dd progress, then enter:

watch -n5 'sudo kill -USR1 $(pgrep ^dd$)'

watch will probe the dd process every -n seconds (-n5 = 5 seconds) and report without halting it.

Note the proper single quotes in the commands above.

26

A few handy sample usages with pv and less typing or more progress then other answers:

First you will need to install pv, with the command:

sudo apt-get install pv

Then some examples are:

pv -n /dev/urandom | dd of=/dev/null
pv -tpreb source.iso | dd of=/dev/BLABLA bs=4096 conv=notrunc,noerror

Note: the first sample is 5 characters less typing then dd if=/dev/urandom | pv | dd of=/dev/null.

And my favorite for cloning a disk drive (replace X with drive letters):

(pv -n /dev/sdX | dd of=/dev/sdX bs=128M conv=notrunc,noerror) 2>&1 | dialog --gauge "Running dd command (cloning), please wait..." 10 70 0

source:

Also for archiving myself.

7

For the sake of completeness:

Version 8.24 of the GNU coreutils includes a patch for dd introducing a parameter to print the progress.

The commit introducing this change has the comment:

dd: new status=progress level to print stats periodically

Many distributions, including Ubuntu 16.04.2 LTS use this version.

4

Use Ctrl+Shift+T while dd is running, and it will output the progress (in bytes):

load: 1.51  cmd: dd 31215 uninterruptible 0.28u 3.67s
321121+0 records in
321120+0 records out
164413440 bytes transferred in 112.708791 secs (1458745 bytes/sec)
6

The best is using it is easy to install through apt-get

4

Native progress status was added to dd!!!

The new version of Coreutils (8.24) adds a progress status to the dd tool:

Usage on Xubuntu 15.10:

Open a terminal and type these commands:

wget ftp://
tar -xf coreutils-8.24.tar.xz
cd coreutils-8.24
./configure && make -j $(nproc)

Run dd as root:

sudo su
cd src
./dd if=/dev/sdc of=/dev/sda conv=noerror status=progress

You will see: Bytes, seconds and speed (Bytes/second).

To check the versions of dd:

Native:

dd --version

New:

cd coreutils-8.24/src
./dd --version

If you have already started dd, and if you are writing a file such as when creating a copy of a pendrive to disk, you can use the watch command to constantly observe the size of the output file to see changes and estimate completion.

watch ls -l /pathtofile/filename

To see only file size (h-human view):

watch ls -sh /pathtofile/filename
3

The dd | pv | dd triad made my 50GB vm copy take 800 seconds, as opposed to 260 seconds using just dd. With this pipeline, at least, pv has no idea how big the input file is so it won't be able to tell you how far along you are so there's no disadvantage to doing it as follows- and you get a nice speed advantage:

I would avoid pv on anything large, and (if using Bash):

Control-Z the dd process

bg to put it in background. Observe that bg will give you output like [1] 6011 where the latter number is a process id. So, do:

while true; do kill -USR1 process_id ; sleep 5; done

where process_id is the process id you observed. Hit Control-C when you see something like:

[1]+  Done dd if=/path/file.qcow2 of=/dev/kvm/pxetest bs=4194304 conv=sparse
-bash: kill: (60111) - No such process

You are done.

Edit: Silly Systems Administrator! Automate your life, don't work! If I have a long dd process that I want to monitor, here's a one-liner that will take care of the whole enchilada for you; put this all on one line:

 dd if=/path/to/bigimage of=/path/to/newimage conv=sparse bs=262144 & bgid=$!; while true; do sleep 1; kill -USR1 $bgid || break; sleep 4; done

You can, of course, script it, perhaps make $1 your input file and $2 your output file. This is left as an exercise for the reader. Note that you need that little sleep before the kill or the kill may die trying to send a signal to dd when it's not ready yet. Adjust your sleeps as desired (maybe even remove the second sleep altogether).

Bash- FTW! :-)

6

Basically:

kill -USR1 < dd pid >
1

On Ubuntu 16.04

Ubuntu 16.04 comes with dd (coreutils) Version 8.25 . Hence the option status=progress is Supported :-)

To use it, just add status=progress along with your dd command.

Example :

dd bs=4M if=/media/severus/tools-soft/OperatingSystems/ubuntu-16.04-desktop-amd64.iso of=/dev/null status=progress && sync

Gives the status as

1282846183 bytes (1.2 GiB, 1.1 GiB) copied, 14.03 s, 101.9 MB/s
James H. Sterling

James H. Sterling

Environmental Science & Climate Journalist

James Sterling reports on renewable energy developments, climate policy, ecological conservation, and green tech innovations around the globe.