Commands tagged progress sorted by votes

Commands tagged progress (28)

sorted by

copy working directory and compress it on-the-fly while showing progress

What happens here is we tell tar to create "-c" an archive of all files in current dir "." (recursively) and output the data to stdout "-f -". Next we specify the size "-s" to pv of all files in current dir. The "du -sb . | awk ?{print $1}?" returns number of bytes in current dir, and it gets fed as "-s" parameter to pv. Next we gzip the whole content and output the result to out.tgz file. This way "pv" knows how much data is still left to be processed and shows us that it will take yet another 4 mins 49 secs to finish. Credit: Peteris Krumins http://www.catonmat.net/blog/unix-utilities-pipe-viewer/ Show Sample Output
This is sample output - yours may be different.
```
253MB 0:00:05 [46.7MB/s] [>     ]  1% ETA 0:04:49
```
26

tar -cf - . | pv -s $(du -sb . | awk '{print $1}') | gzip > out.tgz

opertinicy · 2009-12-18 17:09:08 3
check the status of 'dd' in progress (OS X)

Sends SIGINFO to the process. This is a BSD feature OS X inherited. You must have the terminal window executing dd selected when entering CTRL + T for this to work. Show Sample Output
This is sample output - yours may be different.
```
load: 1.24  cmd: dd 4960 uninterruptible 0.26u 4.59s
476897+0 records in
476896+0 records out
244170752 bytes transferred in 323.318769 secs (755201 bytes/sec)
```
25

CTRL + T

unixmonkey44467 · 2012-12-19 02:21:41 1
Copy with progress
This is sample output - yours may be different.
15

rsync --progress file1 file2

fletch · 2010-02-25 19:57:01 1

check the status of 'dd' in progress (OS X)

"killall -USR1 dd" does not work in OS X for me. However, sending INFO instead of USR1 works. Show Sample Output

killall -INFO dd

jearsh · 2010-04-22 18:38:37 0

Start dd and show progress every X seconds
This is sample output - yours may be different.
12

dd if=/path/inputfile | pv | dd of=/path/outpufile

lucafaus · 2010-12-02 18:11:42 2

send DD a signal to print its progress

every 1sec sends DD the USR1 signal which causes DD to print its progress. Show Sample Output

while :;do killall -USR1 dd;sleep 1;done

oernii2 · 2010-04-07 09:23:31 2

Print one . instead of each line

If you're running a command with a lot of output, this serves as a simple progress indicator. This avoids the need to use `/dev/null` for silencing. It works for any command that outputs lines, updates live (`fflush` avoids buffering), and is simple to understand. Show Sample Output
This is sample output - yours may be different.
```
$ ls ~ | ...
..............
```
7

alias ...="awk '{fflush(); printf \".\"}' && echo \"\""

lgarron · 2014-02-22 22:20:22 4
Check how far along (in %) your program is in a file

Imagine you've started a long-running process that involves piping data, but you forgot to add the progress-bar option to a command. e.g. xz -dc bigdata.xz | complicated-processing-program > summary . This command uses lsof to see how much data xz has read from the file. lsof -o0 -o -Fo FILENAME Display offsets (-o), in decimal (-o0), in parseable form (-Fo) This will output something like: . p12607 f3 o0t45187072 . Process id (p), File Descriptor (f), Offset (o) . We stat the file to get its size stat -c %s FILENAME . Then we plug the values into awk. Split the line at the letter t: -Ft Define a variable for the file's size: -s=$(stat...) Only work on the offset line: /^o/ . Note this command was tested using the Linux version of lsof. Because it uses lsof's batch option (-F) it may be portable. . Thanks to @unhammer for the brilliant idea. Show Sample Output
This is sample output - yours may be different.
```
24%
```
7

F=bigdata.xz; lsof -o0 -o -Fo $F | awk -Ft -v s=$(stat -c %s $F) '/^o/{printf("%d%%\n", 100*$2/s)}'

flatcap · 2015-09-19 22:22:43 1
check the status of 'dd' in progress (OS X)

Your platform may not have pv by default. If you are using Homebew on OSX, simply 'brew install pv'. Show Sample Output
This is sample output - yours may be different.
```
675MiB 0:02:53 [ 4.2MiB/s] [=======>                               ]  9% ETA 0:27:00
```
4

pv -tpreb /dev/sda | dd of=/dev/sdb bs=1M

sc0ttyd · 2013-08-19 23:04:15 1

Start dd and show progress every X seconds

Adjust "sleep X" to your needs. *NOTE: First sleep is required because bash doesn't have a "post-test" syntax (do XXX while). Show Sample Output

dd if=/path/to/inputfile of=/path/to/outputfile & pid=$! && sleep X && while kill -USR1 $pid; do sleep X; done

cyrusza · 2010-12-02 15:07:18 3

Copy a file using dc3dd and watch its progress (very nice alternative to dd)

Requires the dc3dd package - available at http://dc3dd.sourceforge.net Show Sample Output

dc3dd progress=on bs=512 count=2048 if=/dev/zero of=/dev/null

ubahmapk · 2010-01-12 22:54:27 0

See multiple progress bars at once for multiple pipes with pv

In this example we convert a .tar.bz2 file to a .tar.gz file. If you don't have Pipe Viewer, you'll have to download it via apt-get install pv, etc. Show Sample Output
This is sample output - yours may be different.
```
bzcat: 11.4MB 0:00:03 [2.95MB/s] [     <=>                                          ]
 gzip: 4.03MB 0:00:03 [1.05MB/s] [       <=>                                        ]
 orig: 3.12MB 0:00:03 [ 924kB/s] [================>                 ] 45% ETA 0:00:03
```
2

pv -cN orig < foo.tar.bz2 | bzcat | pv -cN bzcat | gzip -9 | pv -cN gzip > foo.tar.gz

rkulla · 2010-04-16 05:21:10 0
Watch the progress of 'dd'

Only slightly different than previous commands. The benefit is that your "watch" should die when the dd command has completed. (Of course this would depend on /proc being available)
This is sample output - yours may be different.
2

dd if=/dev/urandom of=file.img bs=4KB& pid=$!; while [[ -d /proc/$pid ]]; do kill -USR1 $pid && sleep 1 && clear; done

unixmonkey14934 · 2011-06-24 21:49:10 2
send DD a signal to print its progress

Sends the "USR1" signal every 1 second (-n 1) to a process called exactly "dd". The signal in some systems can be INFO or SIGINFO ... look at the signals list in: man kill
This is sample output - yours may be different.
2

watch -n 1 pkill -USR1 "^dd$"

ivanalejandro0 · 2012-08-31 05:15:45 0
Watch the progress of 'dd'

The previously-posted one-liner didn't work for me for whatever reason, so I ended up doing this instead.
This is sample output - yours may be different.
1

dd if=/dev/urandom of=file.img bs=4KB& sleep 1 && pid=`pidof dd`; while [[ -d /proc/$pid ]]; do kill -USR1 $pid && sleep 10 && clear; done

unixmonkey31714 · 2012-02-23 01:45:53 0
Copy with progress
This is sample output - yours may be different.
1

gcp [source] [destination]

vg67 · 2013-07-29 18:32:37 0
check the status of 'dd' in progress (OS X)

While a dd is running in one terminal, open another and enter the while loop. The sample output will be displayed in the window running the dd and the while loop will exit when the dd is complete. It's possible that a "sudo" will need to be inserted before "pkill", depending on your setup, for example: while pgrep ^dd; do sudo pkill -INFO dd; sleep 10; done Show Sample Output
This is sample output - yours may be different.
```
2427+0 records in
2426+0 records out
2543845376 bytes transferred in 313.465401 secs (8115235 bytes/sec)
```
1

while pgrep ^dd; do pkill -INFO dd; sleep 10; done

sternebrau · 2014-01-28 03:09:25 0
Print one . instead of each line

this version only uses shell builtins
This is sample output - yours may be different.
1

alias ...='while read line; do echo -n "."; done && echo ""'

fpunktk · 2014-03-17 17:34:19 0
Command to import Mysql database with a progress bar.
This is sample output - yours may be different.
0

pv -t -p /path/to/sqlfile.sql | mysql -uUSERNAME -pPASSWORD -D DATABASE_NAME

Koobiac · 2011-06-27 13:27:48 0
make rsync progress output suitable for shell script reading

This line unbuffers the interactive output of rsync's --progress flag creating a new line for every update. This output can now be used within a script to make actions (or possibly piped into a GUI generator for a progress bar)
This is sample output - yours may be different.
0

rsync --progress user@host:/path/to/source /path/to/target/ | stdbuf -oL tr '\r' '\n' >> rsyncprogress.txt

stew_rt · 2013-03-26 11:06:45 0
rsync progress indicator which updates in-place

update the output of rsync after completing a file but don't create newlines, just overwrite the last line, this looks a lot better in scripts where you do want to see a progress-indicator, but not the lengthy logs this option is available since rsync 3.1.0 Show Sample Output
This is sample output - yours may be different.
```
          1,696 100%  972.66kB/s    0:00:00 (xfr#1, to-chk=63/65)
```
0

rsync --recursive --info=progress2 <src> <dst>

koter84 · 2014-10-21 22:19:44 0
check the status of 'dd' in progress (OS X)

display dd status on OSX (coreutils) every 10 seconds Show Sample Output
This is sample output - yours may be different.
```
664+0 records in
664+0 records out
696254464 bytes (696 MB) copied, 426.767 s, 1.6 MB/s
```
0

watch -n 10 sudo kill -INFO $(pgrep -l '^dd$' | cut -d ' ' -f 1)

nfarrar · 2014-11-23 21:21:08 0
check the status of 'dd' in progress (OS X)

Observe the process of your dd command on Mac the Mac bash when you burn an .iso or .img on a SD-Card. This command avoids permission errors.
This is sample output - yours may be different.
0

while pgrep ^dd; do sudo pkill -INFO dd; sleep 20; done

kassnik · 2015-12-06 20:16:34 0
transfer lvm image to another host

1. Sends an email to announce the start of the command 2. Raw data copy of block device over ssh to a new host 3. Sends an email to announce the end of the command Make sure the lvm device at the other side exists and has the same amount of logical units.
This is sample output - yours may be different.
0

echo "dd start" | mail -s ready pete@example.com; dd if=/dev/system/machine-disk | ssh -c blowfish root@destination.example.com dd of=/dev/system/machine-disk ; echo "dd ready" | mail -s ready pete@example.com

mipmip · 2016-03-01 13:13:20 0
Copy a file using dd and watch its progress

Coreutils 8.24 added status=progress
This is sample output - yours may be different.
0

dd if=foo of=bar status=progress

ptman · 2016-04-15 08:43:40 0
1 2 >

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Check These Out

Netstat Connection Check

This command does a tally of concurrent active connections from single IPs and prints out those IPs that have the most active concurrent connections. VERY useful in determining the source of a DoS or DDoS attack.

extracts 64 bytes of random digits from random lines out of /dev/random sent to stdio

Use this the next time you need to come up with a reasonably random bitstring, like for a WPA/WPA2 PSK or something. Takes a continuous stream of bytes coming from /dev/urandom, runs it through od(1), picking a random field ($0 and $1 excluded) from a random line and then prints it.

sleep for 1/10s or 1/100s or even 1/1000000s

sleep in microseconds instead of seconds Alternatively to usleep, which is not defined in POSIX 2008 (though it was defined up to POSIX 2004, and it is evidently available on Linux and other platforms with a history of POSIX compliance), the POSIX 2008 standard defines nanosleep

Remote control for Rhythmbox on an Ubuntu Media PC

Note: you'll want to set up pub-key ssh auth. Gives you a quick means of changing volume/tracks/etc for rhythmbox on a remote machine. E.g.: rc --next # Play next track rc --print-playing # Grab the name rc --volume-down rc --help

Check if filesystem hangs

When a fs hangs and you've just one console, even # ls could be a dangerous command. Simply put a trailing "&" and play safe

Generate random password on Mac OS X

Feel free to put this in your ~/.profile: $ random(){ cat /dev/urandom | env LC_CTYPE=C tr -dc $1 | head -c $2; echo; } Then use it to generate passwords: $ random [:alnum:] 16 Or DNA sequences: $ random ACGT 256

Show whats going on restoring files from a spectrum protect backup

spectrum protect's dsmc command shows file names and total amount of restore. This command shows which files are actually open and their siz in GB and highlights the change to the previous output

Monitor the queries being run by MySQL

Save a file you edited in vim without the needed permissions - (Open)solaris version with RBAC

Rename many files in directories and subdirectories

This is probably overkill, but I have some issues when the directories have spaces in their names. The $ find . -type d -print0 | while read -d $'\0' dir; do xxx; done loops over all the subdirectories in this place, ignoring the white spaces (to some extend). $ cd "$dir"; echo " process $dir"; cd -; goes to the directory and back. It also prints some info to check the progress. $ find . -maxdepth 1 -name "*.ogg.mp3" -exec rename 's/.ogg.mp3/.mp3/' {} \; renames the file within the current directory. The whole should work with directories and file names that include white spaces.

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

» all commands
» commands with 3 up-votes (commandlinefu3)
» commands with 10 up-votes (commandlinefu10)
» commands tagged progress

Commands tagged progress (28) the last day the last week the last month all time sorted by date votes

What's this?

Check These Out

Stay in the loop…

Commands tagged progress (28)

sorted by