cat /dev/urandom | pv -L 3m | dd bs=1M count=100 iflag=fullblock > /dev/null

Limit the transfer rate and size of data over a pipe

This example will close the pipe after transferring 100MB at a speed of 3MB per second.

6
By: bugmenot
2012-07-29 00:42:16

These Might Interest You

  • -L RATE, --rate-limit RATE Limit the transfer to a maximum of RATE bytes per second. A suffix of "k", "m", "g", or "t" can be added to denote kilobytes (*1024), megabytes, and so on. It must be an integer.


    2
    pv /dev/urandom -L 3m -i 0.3 > /dev/null
    bugmenot · 2012-07-29 00:46:09 0
  • This example will close the pipe after transferring 100MB at a speed of 3MB per second.


    2
    cat /dev/zero | pv -L 3m -Ss 100m > /dev/null
    bugmenot · 2012-12-15 10:17:52 1
  • Pipe Viewer allows you to monitor the progress of a data transfer or command, or to show the time elapsed, among other things. In this use, it limits the transfer rate of the echo command to 10 bytes per second, making your text appear to be typed out in real time as in Hollywood movies. Fun!


    6
    echo "text to be displayed" | pv -qL 10
    journalsquared · 2010-10-21 20:51:51 1
  • Tc is used to configure Traffic Control in the Linux kernel. See man tc for further details. This series of commands will limit the bandwidth of the specified device to the limit you set (300kbit in the second command) to the address you specify.


    0
    tc qdisc add dev <dev> root handle 1: cbq avpkt 1000 bandwidth 100mbit;tc class add dev <dev> parent 1: classid 1:1 cbq rate 300kbit allot 1500 prio 5 bounded isolated;tc filter add dev <dev> parent 1: protocol ip prio 16 u32 match ip dst <ip> flowid 1:1
    din7 · 2010-02-16 01:39:09 0
  • If you have servers on Wide Area Network (WAN), you may experience very long transfer rates due to limited bandwidth and latency. To speed up you transfers you need to compress the data so you will have less to transfer. So the solution is to use a compression tools like gzip or bzip or compress before and after the data transfer. Using ssh "-C" option is not compatible with every ssh version (ssh2 for instance).


    0
    ssh 10.0.0.4 "gzip -c /tmp/backup.sql" |gunzip > backup.sql
    ultips · 2012-01-06 17:44:06 0
  • This command utilizes 'pv' to show dd's progress. Notes on use with dd: -- dd block size (bs=...) is a widely debated command-line switch and should usually be between 1024 and 4096. You won't see much performance improvements beyond 4096, but regardless of the block size, dd will transfer every bit of data. -- pv's switch, '-s' should be as close to the size of the data source as possible. -- dd's out file, 'of=...' can be anything as the data within that file are the same regardless of the filename / extension. Show Sample Output


    19
    sudo dd if=/dev/sdc bs=4096 | pv -s 2G | sudo dd bs=4096 of=~/USB_BLACK_BACKUP.IMG
    BruceLEET · 2010-07-28 22:39:46 4

What Others Think

'pv' is the 'pipe viewer' written by Andrew Wood (http://www.ivarch.com/programs/pv.shtml) and not the 'phase vocoder' fromt the WAON project included with older Debian and Fedora repositories. For file transfers over TCP/IP, I prefer to use 'rsync' with '--bwlimit'
Mozai · 301 weeks and 5 days ago
pv is more generic than rsync and easier to use.
bugmenot · 300 weeks and 4 days ago

What do you think?

Any thoughts on this command? Does it work on your machine? Can you do the same thing with only 14 characters?

You must be signed in to comment.

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands



Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: