Commands tagged progress sorted by votes

Commands tagged progress (28)

sorted by

copy working directory and compress it on-the-fly while showing progress

What happens here is we tell tar to create "-c" an archive of all files in current dir "." (recursively) and output the data to stdout "-f -". Next we specify the size "-s" to pv of all files in current dir. The "du -sb . | awk ?{print $1}?" returns number of bytes in current dir, and it gets fed as "-s" parameter to pv. Next we gzip the whole content and output the result to out.tgz file. This way "pv" knows how much data is still left to be processed and shows us that it will take yet another 4 mins 49 secs to finish. Credit: Peteris Krumins http://www.catonmat.net/blog/unix-utilities-pipe-viewer/ Show Sample Output
This is sample output - yours may be different.
```
253MB 0:00:05 [46.7MB/s] [>     ]  1% ETA 0:04:49
```
27

tar -cf - . | pv -s $(du -sb . | awk '{print $1}') | gzip > out.tgz

opertinicy · 2009-12-18 17:09:08 28
check the status of 'dd' in progress (OS X)

Sends SIGINFO to the process. This is a BSD feature OS X inherited. You must have the terminal window executing dd selected when entering CTRL + T for this to work. Show Sample Output
This is sample output - yours may be different.
```
load: 1.24  cmd: dd 4960 uninterruptible 0.26u 4.59s
476897+0 records in
476896+0 records out
244170752 bytes transferred in 323.318769 secs (755201 bytes/sec)
```
25

CTRL + T

unixmonkey44467 · 2012-12-19 02:21:41 24

check the status of 'dd' in progress (OS X)

"killall -USR1 dd" does not work in OS X for me. However, sending INFO instead of USR1 works. Show Sample Output

killall -INFO dd

jearsh · 2010-04-22 18:38:37 8

Copy with progress
This is sample output - yours may be different.
15

rsync --progress file1 file2

fletch · 2010-02-25 19:57:01 18
Start dd and show progress every X seconds
This is sample output - yours may be different.
12

dd if=/path/inputfile | pv | dd of=/path/outpufile

lucafaus · 2010-12-02 18:11:42 6

send DD a signal to print its progress

every 1sec sends DD the USR1 signal which causes DD to print its progress. Show Sample Output

while :;do killall -USR1 dd;sleep 1;done

oernii2 · 2010-04-07 09:23:31 10

Print one . instead of each line

If you're running a command with a lot of output, this serves as a simple progress indicator. This avoids the need to use `/dev/null` for silencing. It works for any command that outputs lines, updates live (`fflush` avoids buffering), and is simple to understand. Show Sample Output
This is sample output - yours may be different.
```
$ ls ~ | ...
..............
```
7

alias ...="awk '{fflush(); printf \".\"}' && echo \"\""

lgarron · 2014-02-22 22:20:22 16
Check how far along (in %) your program is in a file

Imagine you've started a long-running process that involves piping data, but you forgot to add the progress-bar option to a command. e.g. xz -dc bigdata.xz | complicated-processing-program > summary . This command uses lsof to see how much data xz has read from the file. lsof -o0 -o -Fo FILENAME Display offsets (-o), in decimal (-o0), in parseable form (-Fo) This will output something like: . p12607 f3 o0t45187072 . Process id (p), File Descriptor (f), Offset (o) . We stat the file to get its size stat -c %s FILENAME . Then we plug the values into awk. Split the line at the letter t: -Ft Define a variable for the file's size: -s=$(stat...) Only work on the offset line: /^o/ . Note this command was tested using the Linux version of lsof. Because it uses lsof's batch option (-F) it may be portable. . Thanks to @unhammer for the brilliant idea. Show Sample Output
This is sample output - yours may be different.
```
24%
```
7

F=bigdata.xz; lsof -o0 -o -Fo $F | awk -Ft -v s=$(stat -c %s $F) '/^o/{printf("%d%%\n", 100*$2/s)}'

flatcap · 2015-09-19 22:22:43 18
check the status of 'dd' in progress (OS X)

Your platform may not have pv by default. If you are using Homebew on OSX, simply 'brew install pv'. Show Sample Output
This is sample output - yours may be different.
```
675MiB 0:02:53 [ 4.2MiB/s] [=======>                               ]  9% ETA 0:27:00
```
4

pv -tpreb /dev/sda | dd of=/dev/sdb bs=1M

sc0ttyd · 2013-08-19 23:04:15 11

Start dd and show progress every X seconds

Adjust "sleep X" to your needs. *NOTE: First sleep is required because bash doesn't have a "post-test" syntax (do XXX while). Show Sample Output

dd if=/path/to/inputfile of=/path/to/outputfile & pid=$! && sleep X && while kill -USR1 $pid; do sleep X; done

cyrusza · 2010-12-02 15:07:18 8

Copy a file using dc3dd and watch its progress (very nice alternative to dd)

Requires the dc3dd package - available at http://dc3dd.sourceforge.net Show Sample Output

dc3dd progress=on bs=512 count=2048 if=/dev/zero of=/dev/null

ubahmapk · 2010-01-12 22:54:27 5

See multiple progress bars at once for multiple pipes with pv

In this example we convert a .tar.bz2 file to a .tar.gz file. If you don't have Pipe Viewer, you'll have to download it via apt-get install pv, etc. Show Sample Output
This is sample output - yours may be different.
```
bzcat: 11.4MB 0:00:03 [2.95MB/s] [     <=>                                          ]
 gzip: 4.03MB 0:00:03 [1.05MB/s] [       <=>                                        ]
 orig: 3.12MB 0:00:03 [ 924kB/s] [================>                 ] 45% ETA 0:00:03
```
2

pv -cN orig < foo.tar.bz2 | bzcat | pv -cN bzcat | gzip -9 | pv -cN gzip > foo.tar.gz

rkulla · 2010-04-16 05:21:10 5
Watch the progress of 'dd'

Only slightly different than previous commands. The benefit is that your "watch" should die when the dd command has completed. (Of course this would depend on /proc being available)
This is sample output - yours may be different.
2

dd if=/dev/urandom of=file.img bs=4KB& pid=$!; while [[ -d /proc/$pid ]]; do kill -USR1 $pid && sleep 1 && clear; done

unixmonkey14934 · 2011-06-24 21:49:10 8
send DD a signal to print its progress

Sends the "USR1" signal every 1 second (-n 1) to a process called exactly "dd". The signal in some systems can be INFO or SIGINFO ... look at the signals list in: man kill
This is sample output - yours may be different.
2

watch -n 1 pkill -USR1 "^dd$"

ivanalejandro0 · 2012-08-31 05:15:45 5
Watch the progress of 'dd'

The previously-posted one-liner didn't work for me for whatever reason, so I ended up doing this instead.
This is sample output - yours may be different.
1

dd if=/dev/urandom of=file.img bs=4KB& sleep 1 && pid=`pidof dd`; while [[ -d /proc/$pid ]]; do kill -USR1 $pid && sleep 10 && clear; done

unixmonkey31714 · 2012-02-23 01:45:53 4
Copy with progress
This is sample output - yours may be different.
1

gcp [source] [destination]

vg67 · 2013-07-29 18:32:37 7
check the status of 'dd' in progress (OS X)

While a dd is running in one terminal, open another and enter the while loop. The sample output will be displayed in the window running the dd and the while loop will exit when the dd is complete. It's possible that a "sudo" will need to be inserted before "pkill", depending on your setup, for example: while pgrep ^dd; do sudo pkill -INFO dd; sleep 10; done Show Sample Output
This is sample output - yours may be different.
```
2427+0 records in
2426+0 records out
2543845376 bytes transferred in 313.465401 secs (8115235 bytes/sec)
```
1

while pgrep ^dd; do pkill -INFO dd; sleep 10; done

sternebrau · 2014-01-28 03:09:25 7
Print one . instead of each line

this version only uses shell builtins
This is sample output - yours may be different.
1

alias ...='while read line; do echo -n "."; done && echo ""'

fpunktk · 2014-03-17 17:34:19 8
Command to import Mysql database with a progress bar.
This is sample output - yours may be different.
0

pv -t -p /path/to/sqlfile.sql | mysql -uUSERNAME -pPASSWORD -D DATABASE_NAME

Koobiac · 2011-06-27 13:27:48 8
make rsync progress output suitable for shell script reading

This line unbuffers the interactive output of rsync's --progress flag creating a new line for every update. This output can now be used within a script to make actions (or possibly piped into a GUI generator for a progress bar)
This is sample output - yours may be different.
0

rsync --progress user@host:/path/to/source /path/to/target/ | stdbuf -oL tr '\r' '\n' >> rsyncprogress.txt

stew_rt · 2013-03-26 11:06:45 7
rsync progress indicator which updates in-place

update the output of rsync after completing a file but don't create newlines, just overwrite the last line, this looks a lot better in scripts where you do want to see a progress-indicator, but not the lengthy logs this option is available since rsync 3.1.0 Show Sample Output
This is sample output - yours may be different.
```
          1,696 100%  972.66kB/s    0:00:00 (xfr#1, to-chk=63/65)
```
0

rsync --recursive --info=progress2 <src> <dst>

koter84 · 2014-10-21 22:19:44 11
check the status of 'dd' in progress (OS X)

display dd status on OSX (coreutils) every 10 seconds Show Sample Output
This is sample output - yours may be different.
```
664+0 records in
664+0 records out
696254464 bytes (696 MB) copied, 426.767 s, 1.6 MB/s
```
0

watch -n 10 sudo kill -INFO $(pgrep -l '^dd$' | cut -d ' ' -f 1)

nfarrar · 2014-11-23 21:21:08 8
check the status of 'dd' in progress (OS X)

Observe the process of your dd command on Mac the Mac bash when you burn an .iso or .img on a SD-Card. This command avoids permission errors.
This is sample output - yours may be different.
0

while pgrep ^dd; do sudo pkill -INFO dd; sleep 20; done

kassnik · 2015-12-06 20:16:34 10
transfer lvm image to another host

1. Sends an email to announce the start of the command 2. Raw data copy of block device over ssh to a new host 3. Sends an email to announce the end of the command Make sure the lvm device at the other side exists and has the same amount of logical units.
This is sample output - yours may be different.
0

echo "dd start" | mail -s ready pete@example.com; dd if=/dev/system/machine-disk | ssh -c blowfish root@destination.example.com dd of=/dev/system/machine-disk ; echo "dd ready" | mail -s ready pete@example.com

mipmip · 2016-03-01 13:13:20 12
Copy a file using dd and watch its progress

Coreutils 8.24 added status=progress
This is sample output - yours may be different.
0

dd if=foo of=bar status=progress

ptman · 2016-04-15 08:43:40 12
1 2 >

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Check These Out

Daemonize nc - Transmit a file like a http server

Allow to launch nc like a daemon, in background until you still stop it. You can stop it with kill %1 (jobs method) or kill PID. The -k option can force nc to listen another connection, but if you use redirection, it will work only one time. The loop's inside doesn't do anything, but we can imagine to send a message to screen when a connection is established

Mount a partition from within a complete disk dump

Suppose you made a backup of your hard disk with dd: dd if=/dev/sda of=/mnt/disk/backup.img This command enables you to mount a partition from inside this image, so you can access your files directly. Substitute PARTITION=1 with the number of the partition you want to mount (returned from sfdisk -d yourfile.img).

Get a facebook likes quantity from CLI

Replace the with your URL, for example http://rublacklist.net/12348/ and it will show likes number

Find where a kind of file is stored

In this case searches for where .desktop files are stored. The resulted is a sorted list of the top directories containing such files.

Create a mirror of a local folder, on a remote server

Create a exact mirror of the local folder "/root/files", on remote server 'remote_server' using SSH command (listening on port 22) (all files & folders on destination server/folder will be deleted)

How To Get the Apache Document Root

Use find to get around Argument list too long problem

Can be used for other commands as well, replace rm with ls. It is easy to make this shorter but if the filenames involved have spaces, you will need to do use find's "-print0" option in conjunction with xargs's "-0" option. Otherwise the shell that xargs uses to execute the "rm" command line will treat the space as a token separator, thereby treating the name as two (or more) names.

Get your external IP address if your machine has a DNS entry

Using DynDNS or a similar service not only allows access to your home machine from outside without needing to know what IP the ISP has assigned to it but it also comes in handy if you want to know your external IP address. The only purpose of the sed command is to remove the leading "host.na.me has address " part from the output. If you don't need to discard it you can simply use $ host $HOSTNAME

Avoid killing the X server with CTRL+C on the tty it was started from

c_rehash replacement

When you don't have c_rehash handy. Really simple - if you have a .pem file that doesn't really contain a x509 cert (let's say, newreq.pem), it will create a link, simply called '.0', pointing to that file.

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

» all commands
» commands with 3 up-votes (commandlinefu3)
» commands with 10 up-votes (commandlinefu10)
» commands tagged progress

Commands tagged progress (28) the last day the last week the last month all time sorted by date votes

What's this?

Check These Out

Stay in the loop…

Commands tagged progress (28)

sorted by