Commands tagged progress sorted by votes

Commands tagged progress (28)

sorted by

copy working directory and compress it on-the-fly while showing progress

What happens here is we tell tar to create "-c" an archive of all files in current dir "." (recursively) and output the data to stdout "-f -". Next we specify the size "-s" to pv of all files in current dir. The "du -sb . | awk ?{print $1}?" returns number of bytes in current dir, and it gets fed as "-s" parameter to pv. Next we gzip the whole content and output the result to out.tgz file. This way "pv" knows how much data is still left to be processed and shows us that it will take yet another 4 mins 49 secs to finish. Credit: Peteris Krumins http://www.catonmat.net/blog/unix-utilities-pipe-viewer/ Show Sample Output
This is sample output - yours may be different.
```
253MB 0:00:05 [46.7MB/s] [>     ]  1% ETA 0:04:49
```
27

tar -cf - . | pv -s $(du -sb . | awk '{print $1}') | gzip > out.tgz

opertinicy · 2009-12-18 17:09:08 9
check the status of 'dd' in progress (OS X)

Sends SIGINFO to the process. This is a BSD feature OS X inherited. You must have the terminal window executing dd selected when entering CTRL + T for this to work. Show Sample Output
This is sample output - yours may be different.
```
load: 1.24  cmd: dd 4960 uninterruptible 0.26u 4.59s
476897+0 records in
476896+0 records out
244170752 bytes transferred in 323.318769 secs (755201 bytes/sec)
```
25

CTRL + T

unixmonkey44467 · 2012-12-19 02:21:41 14

check the status of 'dd' in progress (OS X)

"killall -USR1 dd" does not work in OS X for me. However, sending INFO instead of USR1 works. Show Sample Output

killall -INFO dd

jearsh · 2010-04-22 18:38:37 3

Copy with progress
This is sample output - yours may be different.
15

rsync --progress file1 file2

fletch · 2010-02-25 19:57:01 6
Start dd and show progress every X seconds
This is sample output - yours may be different.
12

dd if=/path/inputfile | pv | dd of=/path/outpufile

lucafaus · 2010-12-02 18:11:42 4

send DD a signal to print its progress

every 1sec sends DD the USR1 signal which causes DD to print its progress. Show Sample Output

while :;do killall -USR1 dd;sleep 1;done

oernii2 · 2010-04-07 09:23:31 7

Print one . instead of each line

If you're running a command with a lot of output, this serves as a simple progress indicator. This avoids the need to use `/dev/null` for silencing. It works for any command that outputs lines, updates live (`fflush` avoids buffering), and is simple to understand. Show Sample Output
This is sample output - yours may be different.
```
$ ls ~ | ...
..............
```
7

alias ...="awk '{fflush(); printf \".\"}' && echo \"\""

lgarron · 2014-02-22 22:20:22 11
Check how far along (in %) your program is in a file

Imagine you've started a long-running process that involves piping data, but you forgot to add the progress-bar option to a command. e.g. xz -dc bigdata.xz | complicated-processing-program > summary . This command uses lsof to see how much data xz has read from the file. lsof -o0 -o -Fo FILENAME Display offsets (-o), in decimal (-o0), in parseable form (-Fo) This will output something like: . p12607 f3 o0t45187072 . Process id (p), File Descriptor (f), Offset (o) . We stat the file to get its size stat -c %s FILENAME . Then we plug the values into awk. Split the line at the letter t: -Ft Define a variable for the file's size: -s=$(stat...) Only work on the offset line: /^o/ . Note this command was tested using the Linux version of lsof. Because it uses lsof's batch option (-F) it may be portable. . Thanks to @unhammer for the brilliant idea. Show Sample Output
This is sample output - yours may be different.
```
24%
```
7

F=bigdata.xz; lsof -o0 -o -Fo $F | awk -Ft -v s=$(stat -c %s $F) '/^o/{printf("%d%%\n", 100*$2/s)}'

flatcap · 2015-09-19 22:22:43 7
check the status of 'dd' in progress (OS X)

Your platform may not have pv by default. If you are using Homebew on OSX, simply 'brew install pv'. Show Sample Output
This is sample output - yours may be different.
```
675MiB 0:02:53 [ 4.2MiB/s] [=======>                               ]  9% ETA 0:27:00
```
4

pv -tpreb /dev/sda | dd of=/dev/sdb bs=1M

sc0ttyd · 2013-08-19 23:04:15 3

Start dd and show progress every X seconds

Adjust "sleep X" to your needs. *NOTE: First sleep is required because bash doesn't have a "post-test" syntax (do XXX while). Show Sample Output

dd if=/path/to/inputfile of=/path/to/outputfile & pid=$! && sleep X && while kill -USR1 $pid; do sleep X; done

cyrusza · 2010-12-02 15:07:18 6

Copy a file using dc3dd and watch its progress (very nice alternative to dd)

Requires the dc3dd package - available at http://dc3dd.sourceforge.net Show Sample Output

dc3dd progress=on bs=512 count=2048 if=/dev/zero of=/dev/null

ubahmapk · 2010-01-12 22:54:27 3

See multiple progress bars at once for multiple pipes with pv

In this example we convert a .tar.bz2 file to a .tar.gz file. If you don't have Pipe Viewer, you'll have to download it via apt-get install pv, etc. Show Sample Output
This is sample output - yours may be different.
```
bzcat: 11.4MB 0:00:03 [2.95MB/s] [     <=>                                          ]
 gzip: 4.03MB 0:00:03 [1.05MB/s] [       <=>                                        ]
 orig: 3.12MB 0:00:03 [ 924kB/s] [================>                 ] 45% ETA 0:00:03
```
2

pv -cN orig < foo.tar.bz2 | bzcat | pv -cN bzcat | gzip -9 | pv -cN gzip > foo.tar.gz

rkulla · 2010-04-16 05:21:10 2
Watch the progress of 'dd'

Only slightly different than previous commands. The benefit is that your "watch" should die when the dd command has completed. (Of course this would depend on /proc being available)
This is sample output - yours may be different.
2

dd if=/dev/urandom of=file.img bs=4KB& pid=$!; while [[ -d /proc/$pid ]]; do kill -USR1 $pid && sleep 1 && clear; done

unixmonkey14934 · 2011-06-24 21:49:10 6
send DD a signal to print its progress

Sends the "USR1" signal every 1 second (-n 1) to a process called exactly "dd". The signal in some systems can be INFO or SIGINFO ... look at the signals list in: man kill
This is sample output - yours may be different.
2

watch -n 1 pkill -USR1 "^dd$"

ivanalejandro0 · 2012-08-31 05:15:45 2
Watch the progress of 'dd'

The previously-posted one-liner didn't work for me for whatever reason, so I ended up doing this instead.
This is sample output - yours may be different.
1

dd if=/dev/urandom of=file.img bs=4KB& sleep 1 && pid=`pidof dd`; while [[ -d /proc/$pid ]]; do kill -USR1 $pid && sleep 10 && clear; done

unixmonkey31714 · 2012-02-23 01:45:53 2
Copy with progress
This is sample output - yours may be different.
1

gcp [source] [destination]

vg67 · 2013-07-29 18:32:37 2
check the status of 'dd' in progress (OS X)

While a dd is running in one terminal, open another and enter the while loop. The sample output will be displayed in the window running the dd and the while loop will exit when the dd is complete. It's possible that a "sudo" will need to be inserted before "pkill", depending on your setup, for example: while pgrep ^dd; do sudo pkill -INFO dd; sleep 10; done Show Sample Output
This is sample output - yours may be different.
```
2427+0 records in
2426+0 records out
2543845376 bytes transferred in 313.465401 secs (8115235 bytes/sec)
```
1

while pgrep ^dd; do pkill -INFO dd; sleep 10; done

sternebrau · 2014-01-28 03:09:25 3
Print one . instead of each line

this version only uses shell builtins
This is sample output - yours may be different.
1

alias ...='while read line; do echo -n "."; done && echo ""'

fpunktk · 2014-03-17 17:34:19 3
Command to import Mysql database with a progress bar.
This is sample output - yours may be different.
0

pv -t -p /path/to/sqlfile.sql | mysql -uUSERNAME -pPASSWORD -D DATABASE_NAME

Koobiac · 2011-06-27 13:27:48 6
make rsync progress output suitable for shell script reading

This line unbuffers the interactive output of rsync's --progress flag creating a new line for every update. This output can now be used within a script to make actions (or possibly piped into a GUI generator for a progress bar)
This is sample output - yours may be different.
0

rsync --progress user@host:/path/to/source /path/to/target/ | stdbuf -oL tr '\r' '\n' >> rsyncprogress.txt

stew_rt · 2013-03-26 11:06:45 4
rsync progress indicator which updates in-place

update the output of rsync after completing a file but don't create newlines, just overwrite the last line, this looks a lot better in scripts where you do want to see a progress-indicator, but not the lengthy logs this option is available since rsync 3.1.0 Show Sample Output
This is sample output - yours may be different.
```
          1,696 100%  972.66kB/s    0:00:00 (xfr#1, to-chk=63/65)
```
0

rsync --recursive --info=progress2 <src> <dst>

koter84 · 2014-10-21 22:19:44 7
check the status of 'dd' in progress (OS X)

display dd status on OSX (coreutils) every 10 seconds Show Sample Output
This is sample output - yours may be different.
```
664+0 records in
664+0 records out
696254464 bytes (696 MB) copied, 426.767 s, 1.6 MB/s
```
0

watch -n 10 sudo kill -INFO $(pgrep -l '^dd$' | cut -d ' ' -f 1)

nfarrar · 2014-11-23 21:21:08 5
check the status of 'dd' in progress (OS X)

Observe the process of your dd command on Mac the Mac bash when you burn an .iso or .img on a SD-Card. This command avoids permission errors.
This is sample output - yours may be different.
0

while pgrep ^dd; do sudo pkill -INFO dd; sleep 20; done

kassnik · 2015-12-06 20:16:34 6
transfer lvm image to another host

1. Sends an email to announce the start of the command 2. Raw data copy of block device over ssh to a new host 3. Sends an email to announce the end of the command Make sure the lvm device at the other side exists and has the same amount of logical units.
This is sample output - yours may be different.
0

echo "dd start" | mail -s ready pete@example.com; dd if=/dev/system/machine-disk | ssh -c blowfish root@destination.example.com dd of=/dev/system/machine-disk ; echo "dd ready" | mail -s ready pete@example.com

mipmip · 2016-03-01 13:13:20 8
Copy a file using dd and watch its progress

Coreutils 8.24 added status=progress
This is sample output - yours may be different.
0

dd if=foo of=bar status=progress

ptman · 2016-04-15 08:43:40 8
1 2 >

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Check These Out

List the binaries installed by a Debian package

GNU grep's perl-compatible regular expression(PCRE).

Rename files in batch

Recursively remove directory with many files quickly

rsync'ing an empty directory over a directory to be deleted recursively is much faster than using rm -rf, for various reasons. Relevant only for directories with really a lot of files.

List installed deb packages by size

how to export a table in .csv file

Exports the result of query in a csv file

Paged, colored svn diff

I put this in a shell script called "svndiff", as it provides a handy decorated "svn diff" output that is colored (which you can't see here) and paged. The -r is required so less doesn't mangle the color codes.

Swap a file or dir with quick resotre

This lets you replace a file or directory and quickly revert if something goes wrong. For example, the current version of a website's files are in public_html. Put a new version of the site in public_html~ and execute the command. The names are swapped. If anything goes wrong, execute it again (up arrow or !!).

Quicker move to parent directory

Alias two dots to move to parent directory. Put it into your .bashrc or .profile file.

Get AWS temporary credentials ready to export based on a MFA virtual appliance

You might want to secure your AWS operations requiring to use a MFA token. But then to use API or tools, you need to pass credentials generated with a MFA token. This commands asks you for the MFA code and retrieves these credentials using AWS Cli. To print the exports, you can use: `awk '{ print "export AWS_ACCESS_KEY_ID=\"" $1 "\"\n" "export AWS_SECRET_ACCESS_KEY=\"" $2 "\"\n" "export AWS_SESSION_TOKEN=\"" $3 "\"" }'` You must adapt the command line to include: * $MFA_IDis ARN of the virtual MFA or serial number of the physical one * TTL for the credentials

Convert all WMF images to SVG recursively ignoring file extension case

This assumes you have the package installed necessary for converting WMF files. On my Ubuntu box, this is libwmf-bin. I used this command, as libwmf is not on my wife's iMac, so I archived the directories containing the WMF files from OS X, ran them on my Ubuntu box, archived the resulting SVGs, and sent them back to her. Quick, simple and to the point. Searches directories recursively looking for extensions ignoring case. This is much more readable and clean than -exec for find. The while loop also gives further flexibility on complex logic. Also, although there is 'wmf2svg --auto', it expects lowercase extensions, and not uppercase. Because I want to ignore case, I need to use the -o option instead. Works in ZSH and BASH. Haven't tested in other shells.

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

» all commands
» commands with 3 up-votes (commandlinefu3)
» commands with 10 up-votes (commandlinefu10)
» commands tagged progress

Commands tagged progress (28) the last day the last week the last month all time sorted by date votes

What's this?

Check These Out

Stay in the loop…

Commands tagged progress (28)

sorted by