Commands tagged progress sorted by votes

Commands tagged progress (28)

sorted by

copy working directory and compress it on-the-fly while showing progress

What happens here is we tell tar to create "-c" an archive of all files in current dir "." (recursively) and output the data to stdout "-f -". Next we specify the size "-s" to pv of all files in current dir. The "du -sb . | awk ?{print $1}?" returns number of bytes in current dir, and it gets fed as "-s" parameter to pv. Next we gzip the whole content and output the result to out.tgz file. This way "pv" knows how much data is still left to be processed and shows us that it will take yet another 4 mins 49 secs to finish. Credit: Peteris Krumins http://www.catonmat.net/blog/unix-utilities-pipe-viewer/ Show Sample Output
This is sample output - yours may be different.
```
253MB 0:00:05 [46.7MB/s] [>     ]  1% ETA 0:04:49
```
26

tar -cf - . | pv -s $(du -sb . | awk '{print $1}') | gzip > out.tgz

opertinicy · 2009-12-18 17:09:08 3
check the status of 'dd' in progress (OS X)

Sends SIGINFO to the process. This is a BSD feature OS X inherited. You must have the terminal window executing dd selected when entering CTRL + T for this to work. Show Sample Output
This is sample output - yours may be different.
```
load: 1.24  cmd: dd 4960 uninterruptible 0.26u 4.59s
476897+0 records in
476896+0 records out
244170752 bytes transferred in 323.318769 secs (755201 bytes/sec)
```
25

CTRL + T

unixmonkey44467 · 2012-12-19 02:21:41 4
Copy with progress
This is sample output - yours may be different.
15

rsync --progress file1 file2

fletch · 2010-02-25 19:57:01 1

check the status of 'dd' in progress (OS X)

"killall -USR1 dd" does not work in OS X for me. However, sending INFO instead of USR1 works. Show Sample Output

killall -INFO dd

jearsh · 2010-04-22 18:38:37 0

Start dd and show progress every X seconds
This is sample output - yours may be different.
12

dd if=/path/inputfile | pv | dd of=/path/outpufile

lucafaus · 2010-12-02 18:11:42 2

send DD a signal to print its progress

every 1sec sends DD the USR1 signal which causes DD to print its progress. Show Sample Output

while :;do killall -USR1 dd;sleep 1;done

oernii2 · 2010-04-07 09:23:31 2

Print one . instead of each line

If you're running a command with a lot of output, this serves as a simple progress indicator. This avoids the need to use `/dev/null` for silencing. It works for any command that outputs lines, updates live (`fflush` avoids buffering), and is simple to understand. Show Sample Output
This is sample output - yours may be different.
```
$ ls ~ | ...
..............
```
7

alias ...="awk '{fflush(); printf \".\"}' && echo \"\""

lgarron · 2014-02-22 22:20:22 4
Check how far along (in %) your program is in a file

Imagine you've started a long-running process that involves piping data, but you forgot to add the progress-bar option to a command. e.g. xz -dc bigdata.xz | complicated-processing-program > summary . This command uses lsof to see how much data xz has read from the file. lsof -o0 -o -Fo FILENAME Display offsets (-o), in decimal (-o0), in parseable form (-Fo) This will output something like: . p12607 f3 o0t45187072 . Process id (p), File Descriptor (f), Offset (o) . We stat the file to get its size stat -c %s FILENAME . Then we plug the values into awk. Split the line at the letter t: -Ft Define a variable for the file's size: -s=$(stat...) Only work on the offset line: /^o/ . Note this command was tested using the Linux version of lsof. Because it uses lsof's batch option (-F) it may be portable. . Thanks to @unhammer for the brilliant idea. Show Sample Output
This is sample output - yours may be different.
```
24%
```
7

F=bigdata.xz; lsof -o0 -o -Fo $F | awk -Ft -v s=$(stat -c %s $F) '/^o/{printf("%d%%\n", 100*$2/s)}'

flatcap · 2015-09-19 22:22:43 1
check the status of 'dd' in progress (OS X)

Your platform may not have pv by default. If you are using Homebew on OSX, simply 'brew install pv'. Show Sample Output
This is sample output - yours may be different.
```
675MiB 0:02:53 [ 4.2MiB/s] [=======>                               ]  9% ETA 0:27:00
```
4

pv -tpreb /dev/sda | dd of=/dev/sdb bs=1M

sc0ttyd · 2013-08-19 23:04:15 1

Start dd and show progress every X seconds

Adjust "sleep X" to your needs. *NOTE: First sleep is required because bash doesn't have a "post-test" syntax (do XXX while). Show Sample Output

dd if=/path/to/inputfile of=/path/to/outputfile & pid=$! && sleep X && while kill -USR1 $pid; do sleep X; done

cyrusza · 2010-12-02 15:07:18 3

Copy a file using dc3dd and watch its progress (very nice alternative to dd)

Requires the dc3dd package - available at http://dc3dd.sourceforge.net Show Sample Output

dc3dd progress=on bs=512 count=2048 if=/dev/zero of=/dev/null

ubahmapk · 2010-01-12 22:54:27 0

See multiple progress bars at once for multiple pipes with pv

In this example we convert a .tar.bz2 file to a .tar.gz file. If you don't have Pipe Viewer, you'll have to download it via apt-get install pv, etc. Show Sample Output
This is sample output - yours may be different.
```
bzcat: 11.4MB 0:00:03 [2.95MB/s] [     <=>                                          ]
 gzip: 4.03MB 0:00:03 [1.05MB/s] [       <=>                                        ]
 orig: 3.12MB 0:00:03 [ 924kB/s] [================>                 ] 45% ETA 0:00:03
```
2

pv -cN orig < foo.tar.bz2 | bzcat | pv -cN bzcat | gzip -9 | pv -cN gzip > foo.tar.gz

rkulla · 2010-04-16 05:21:10 0
Watch the progress of 'dd'

Only slightly different than previous commands. The benefit is that your "watch" should die when the dd command has completed. (Of course this would depend on /proc being available)
This is sample output - yours may be different.
2

dd if=/dev/urandom of=file.img bs=4KB& pid=$!; while [[ -d /proc/$pid ]]; do kill -USR1 $pid && sleep 1 && clear; done

unixmonkey14934 · 2011-06-24 21:49:10 2
send DD a signal to print its progress

Sends the "USR1" signal every 1 second (-n 1) to a process called exactly "dd". The signal in some systems can be INFO or SIGINFO ... look at the signals list in: man kill
This is sample output - yours may be different.
2

watch -n 1 pkill -USR1 "^dd$"

ivanalejandro0 · 2012-08-31 05:15:45 0
Watch the progress of 'dd'

The previously-posted one-liner didn't work for me for whatever reason, so I ended up doing this instead.
This is sample output - yours may be different.
1

dd if=/dev/urandom of=file.img bs=4KB& sleep 1 && pid=`pidof dd`; while [[ -d /proc/$pid ]]; do kill -USR1 $pid && sleep 10 && clear; done

unixmonkey31714 · 2012-02-23 01:45:53 0
Copy with progress
This is sample output - yours may be different.
1

gcp [source] [destination]

vg67 · 2013-07-29 18:32:37 0
check the status of 'dd' in progress (OS X)

While a dd is running in one terminal, open another and enter the while loop. The sample output will be displayed in the window running the dd and the while loop will exit when the dd is complete. It's possible that a "sudo" will need to be inserted before "pkill", depending on your setup, for example: while pgrep ^dd; do sudo pkill -INFO dd; sleep 10; done Show Sample Output
This is sample output - yours may be different.
```
2427+0 records in
2426+0 records out
2543845376 bytes transferred in 313.465401 secs (8115235 bytes/sec)
```
1

while pgrep ^dd; do pkill -INFO dd; sleep 10; done

sternebrau · 2014-01-28 03:09:25 0
Print one . instead of each line

this version only uses shell builtins
This is sample output - yours may be different.
1

alias ...='while read line; do echo -n "."; done && echo ""'

fpunktk · 2014-03-17 17:34:19 0
Command to import Mysql database with a progress bar.
This is sample output - yours may be different.
0

pv -t -p /path/to/sqlfile.sql | mysql -uUSERNAME -pPASSWORD -D DATABASE_NAME

Koobiac · 2011-06-27 13:27:48 0
make rsync progress output suitable for shell script reading

This line unbuffers the interactive output of rsync's --progress flag creating a new line for every update. This output can now be used within a script to make actions (or possibly piped into a GUI generator for a progress bar)
This is sample output - yours may be different.
0

rsync --progress [email protected]:/path/to/source /path/to/target/ | stdbuf -oL tr '\r' '\n' >> rsyncprogress.txt

stew_rt · 2013-03-26 11:06:45 0
rsync progress indicator which updates in-place

update the output of rsync after completing a file but don't create newlines, just overwrite the last line, this looks a lot better in scripts where you do want to see a progress-indicator, but not the lengthy logs this option is available since rsync 3.1.0 Show Sample Output
This is sample output - yours may be different.
```
          1,696 100%  972.66kB/s    0:00:00 (xfr#1, to-chk=63/65)
```
0

rsync --recursive --info=progress2 <src> <dst>

koter84 · 2014-10-21 22:19:44 0
check the status of 'dd' in progress (OS X)

display dd status on OSX (coreutils) every 10 seconds Show Sample Output
This is sample output - yours may be different.
```
664+0 records in
664+0 records out
696254464 bytes (696 MB) copied, 426.767 s, 1.6 MB/s
```
0

watch -n 10 sudo kill -INFO $(pgrep -l '^dd$' | cut -d ' ' -f 1)

nfarrar · 2014-11-23 21:21:08 0
check the status of 'dd' in progress (OS X)

Observe the process of your dd command on Mac the Mac bash when you burn an .iso or .img on a SD-Card. This command avoids permission errors.
This is sample output - yours may be different.
0

while pgrep ^dd; do sudo pkill -INFO dd; sleep 20; done

kassnik · 2015-12-06 20:16:34 0
transfer lvm image to another host

1. Sends an email to announce the start of the command 2. Raw data copy of block device over ssh to a new host 3. Sends an email to announce the end of the command Make sure the lvm device at the other side exists and has the same amount of logical units.
This is sample output - yours may be different.
0

echo "dd start" | mail -s ready [email protected]; dd if=/dev/system/machine-disk | ssh -c blowfish [email protected] dd of=/dev/system/machine-disk ; echo "dd ready" | mail -s ready [email protected]

mipmip · 2016-03-01 13:13:20 0
Copy a file using dd and watch its progress

Coreutils 8.24 added status=progress
This is sample output - yours may be different.
0

dd if=foo of=bar status=progress

ptman · 2016-04-15 08:43:40 0
1 2 >

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Check These Out

list block devices

Shows all block devices in a tree with descruptions of what they are.

Get your commandlinefu points (upvotes - downvotes)

This version prints current votes and commands for a user. Pass the user as an argument. While this technically "fits" as a one liner, it really is easier to look at as a shell script with extra whitespace. :)

View network activity of any application or user in realtime

The "-r 2" option puts lsof in repeat mode, with updates every 2 seconds. (Ctrl -c quits) The "-p" option is used to specify the application PID you want to monitor. The "-u' option can be used to keep an eye on a users network activity. "lsof -r 2 -u username -i -a"

Install pip with Proxy

Installs pip packages defining a proxy

list block devices

Shows all block devices in a tree with descruptions of what they are.

Backup (archive) your Gmail IMAP folders.

Copies an entire hierarchy of mailboxes from the named POP3/IMAP/etc. source to the named destination. Mailboxes are created on the destination as needed. NOTE: The 'mailutil' is Washington's University 'mailutil' (apt-get install uw-mailutils). More examples $ mailutil transfer {imap.gmail.com/ssl/[email protected]}INBOX Gmail/ ; mailutil check imap.gmail.com/ssl/[email protected]}\[Gmail\]/Spam If you use the utility in the first, append -v|-d flag(s) to the end the commands above (man mailutil).

bash script to zip a folder while ignoring git files and copying it to dropbox

Better tool for exporting git's repository is Git itself!

find duplicate messages in a Maildir

# find assumes email files start with a number 1-9 # sed joins the lines starting with " " to the previous line # gawk print the received and from lines # sort according to the second field (received+from) # uniq print the duplicated filename # a message is viewed as duplicate if it is received at the same time as another message, and from the same person. The command was intended to be run under cron. If run in a terminal, mutt can be used: mutt -e "push otD~=xq" -f $folder

Execute a sudo command remotely, without displaying the password

The ssh command alone will execute the sudo command remotely, but the password will be visible in the terminal as you type it. The two stty commands disable the terminal from echoing the password back to you, which makes the remote sudo act as it does locally.

Port Knocking!

Knock on ports to open a port to a service (ssh for example) and knock again to close the port. You have to install knockd. See example config file below. [options] logfile = /var/log/knockd.log [openSSH] sequence = 3000,4000,5000 seq_timeout = 5 command = /sbin/iptables -A INPUT -i eth0 -s %IP% -p tcp --dport 22 -j ACCEPT tcpflags = syn [closeSSH] sequence = 5000,4000,3000 seq_timeout = 5 command = /sbin/iptables -D INPUT -i eth0 -s %IP% -p tcp --dport 22 -j ACCEPT tcpflags = syn

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

» all commands
» commands with 3 up-votes (commandlinefu3)
» commands with 10 up-votes (commandlinefu10)
» commands tagged progress

Commands tagged progress (28) the last day the last week the last month all time sorted by date votes

What's this?

Check These Out

Stay in the loop…

Commands tagged progress (28)

sorted by