Commands tagged pipe sorted by votes

Commands tagged pipe (23)

sorted by

Broadcast your shell thru ports 5000, 5001, 5002 ...

run 'nc yourip 5000', 'nc yourip 5001' or 'nc yourip 5002' elsewhere will produce an exact same mirror of your shell. This is handy when you want to show someone else some amazing stuff in your shell without giving them control over it.
This is sample output - yours may be different.
41

script -qf | tee >(nc -kl 5000) >(nc -kl 5001) >(nc -kl 5002)

clvv · 2010-10-11 07:55:30 21
Use tee to process a pipe with two or more processes

Tee can be used to split a pipe into multiple streams for one or more process to work it. You can add more " >()" for even more fun. Show Sample Output
This is sample output - yours may be different.
```
tee can split a pipe in two
tee_can_split_a_pipe_in_two
owt ni epip a tilps nac eet
```
34

echo "tee can split a pipe in two"|tee >(rev) >(tr ' ' '_')

axelabs · 2010-08-14 20:38:59 10
GREP a PDF file.

PDF files are simultaneously wonderful and heinous. They are wonderful in being ubiquitous and mostly being cross platform. They are heinous in being very difficult to work with from the command line, search, grep, use only the text inside the PDF, or use outside of proprietary products. xpdf is a wonderful set of PDF tools. It is on many linux distros and can be installed on OS X. While primarily an open PDF viewer for X, xpdf has the tool "pdftotext" that can extract formated or unformatted text from inside a PDF that has text. This text stream can then be further processed by grep or other tool. The '-' after the file name directs output to stdout rather than to a text file the same name as the PDF. Make sure you use version 3.02 of pdftotext or later; earlier versions clipped lines. The lines extracted from a PDF without the "-layout" option are very long. More paragraphs. Use just to test that a pattern exists in the file. With "-layout" the output resembles the lines, but it is not perfect. xpdf is available open source at http://www.foolabs.com/xpdf/
This is sample output - yours may be different.
27

pdftotext [file] - | grep 'YourPattern'

drewk · 2010-02-14 21:42:35 19
Pipe stdout and stderr, etc., to separate commands

You can use [n]> combined with >(cmd) to attach the various output file descriptors to be the input of different commands.
This is sample output - yours may be different.
26

some_command > >(/bin/cmd_for_stdout) 2> >(/bin/cmd_for_stderr)

tylerl · 2009-12-01 03:58:04 16
Duplicate several drives concurrently

If you have some drive imaging to do, you can boot into any liveCD and use a commodity machine. The drives will be written in parallel. To improve efficiency, specify a larger block size in dd: dd if=/dev/sda bs=64k | tee >(dd of=/dev/sdb bs=64k) | dd of=/dev/sdc bs=64k To image more drives , insert them as additional arguments to tee: dd if=/dev/sda | tee >(dd of=/dev/sdb) >(dd of=/dev/sdc) >(dd of=/dev/sdd) | dd of=/dev/sde
This is sample output - yours may be different.
23

dd if=/dev/sda | tee >(dd of=/dev/sdb) | dd of=/dev/sdc

nerd65536 · 2009-12-11 17:34:38 25
Fast, built-in pipe-based data sink

This is shorter and actually much faster than >/dev/null (see sample output for timings) Plus, it looks like a disappointed face emoticon. Show Sample Output
This is sample output - yours may be different.
```
# with >/dev/null:
$ time dd if=/dev/zero bs=1K count=100000 >/dev/null

100000+0 records in
100000+0 records out
102400000 bytes (102 MB) copied, 0.0355178 s, 2.9 GB/s

real    0m0.037s
user    0m0.011s
sys     0m0.025s

# with |:
$ time dd if=/dev/zero bs=1K count=100000 |:

real    0m0.002s
user    0m0.000s
sys     0m0.002s
```
22

<COMMAND> |:

h3xx · 2011-08-28 23:48:29 24
Print text string vertically, one character per line.

Show Sample Output
This is sample output - yours may be different.
```
P
r
i
n
t
 
t
e
x
t
 
v
e
r
t
i
c
a
l
l
y
```
9

echo Print text vertically|sed 's/$.$/\1\n/g'

axelabs · 2009-03-04 23:58:02 16
Clean up the garbage an accidental unzipping makes

List out all the names from the zip file and pass it to xargs utility to delete each one of them
This is sample output - yours may be different.
5

unzip -Z -1 <filename.zip> | xargs -I{} rm -v {}

praton · 2018-08-19 20:49:25 383
Prettify XML in pipeline

xmllint2 comes with GNU libxml2 library. (based on: http://vim.wikia.com/wiki/Format_your_xml_document_using_xmllint) Show Sample Output
This is sample output - yours may be different.
```
<?xml version="1.0"?>
<foo>
  <bar/>
</foo>
```
2

echo '<foo><bar/></foo>' | xmllint --format -

akavel · 2012-01-12 09:39:56 27
GREP a PDF file.

This is a good alternative to pdf2text for Ubuntu. To install it: sudo apt-get install python-pdfminer
This is sample output - yours may be different.
2

pdf2txt myfile.pdf | grep mypattern

grinob · 2015-11-23 17:46:22 9
Throttling Bandwidth On A Mac

sudo ipfw pipe 1 config bw 50KByte/s Set the bandwidth (bw) limit to any number you want. For example you could have a 15kb pipe for X application and then a 100kb pipe for another application and attach things to those pipes. If a port isn’t attached to a pipe, it runs at full speed. Change the number (in this case 1) to a different number for a different pipe. The next step is to attach your port. sudo ipfw add 1 pipe 1 src-port 80 In this case anything on port 80 (http) will be set to a limit of 50Kbyte/s. If you want to attach a second port to this pipe, repeat the command but change the port number at the end. src : http://www.mactricksandtips.com/2008/12/throttling-bandwidth-on-a-mac.html
This is sample output - yours may be different.
1

sudo ipfw pipe 1 config bw 50KByte/s;sudo ipfw add 1 pipe 1 src-port 80

miccaman · 2009-07-02 23:17:49 6
Broadcast your shell thru UDP on port 5000

Better solution in case of many clients, imo. Show Sample Output
This is sample output - yours may be different.
```
For client:
nc -lu -s 192.168.1.255 -p 5000
```
1

script -qf >(nc -ub 192.168.1.255 5000)

Archiater · 2011-04-23 12:38:25 4
GREP a PDF file.
This is sample output - yours may be different.
0

grep -i '[^script$]' 1.txt

vinod · 2010-10-20 12:17:04 4
Full remote server backup over cstream using tar (excluding unnecessary files) (reports every 10 seconds)

The files are automatically uncompressed when they reach the destination machine. This is a fast way to backup your server to your local computer while it's running (shutting down services is recommended). A file named "exclude.txt" is needed at /tmp/ containing the following : /dev/* /media/* /mnt/* /proc/* /sys/* /tmp/* /home/*/.local/share/Trash /home/*/.gvfs /home/*/.cache /home/*/.thumbnails /etc/fstab /lib/modules/*/volatile/.mounted /var/run/* /var/lock/* /var/tmp/* /var/cache/apt/archives/* /lost+found/* Show Sample Output
This is sample output - yours may be different.
```
895643648 B 854.2 MB 2125.2 s (35:25 min) 421438 B/s 411.56 KB/s
898371584 B 856.8 MB 2132.5 s (35:32 min) 421280 B/s 411.41 KB/s
901783552 B 860.0 MB 2139.8 s (35:39 min) 421431 B/s 411.55 KB/s
904785920 B 862.9 MB 2147.0 s (35:46 min) 421420 B/s 411.54 KB/s
908300288 B 866.2 MB 2154.0 s (35:54 min) 421678 B/s 411.79 KB/s
911953920 B 869.7 MB 2161.2 s (36:01 min) 421969 B/s 412.08 KB/s
914890752 B 872.5 MB 2168.4 s (36:08 min) 421929 B/s 412.04 KB/s
919408640 B 876.8 MB 2175.4 s (36:15 min) 422647 B/s 412.74 KB/s
923365376 B 880.6 MB 2182.6 s (36:22 min) 423061 B/s 413.15 KB/s
```
0

tar -cj / -X /tmp/exclude.txt | cstream -v 1 -c 3 -T 10 | ssh user@host 'tar -xj -C /backupDestination'

fantleas · 2014-07-21 18:52:19 9
Watch who requests what page from apache logs

Use this command to watch apache access logs in real time to see what pages are getting hit. Show Sample Output
This is sample output - yours may be different.
```
127.0.0.1 "http://example.com/example-page.html"
127.0.0.1 "http://example.com/index.html"
127.0.0.1 "http://example.com/index.html"
```
0

tail -f access_log | awk '{print $1 , $12}'

tyzbit · 2014-12-24 14:15:52 15
Redirect output to a write-protected file with sudo but without sh -c, using tee.

Use tee -a to append.
This is sample output - yours may be different.
0

command foo bar | sudo tee /etc/write-protected > /dev/null

adeverteuil · 2015-02-08 03:58:35 9
GREP a PDF file.

grep pdf files easily
This is sample output - yours may be different.
0

pdfgrep pattern /the/path

ees · 2017-02-02 09:27:43 19
restore <mysqldump>.tar.gz on the fly

`tar xfzO` extracts to STDOUT which got redirected directly to mysql. Really helpful, when your hard drive can't fit two copies of non-compressed database :)
This is sample output - yours may be different.
-1

tar xfzO <backup_name>.tar.gz | mysql -u root <database_name>

alecnmk · 2011-02-10 22:18:42 6
Broadcast your shell thru ports 5000, 5001, 5002 ...

solution that works for me
This is sample output - yours may be different.
-1

script -qf | tee >(nc -l -p 5000)

nesses · 2013-04-13 00:22:02 6
Fast, built-in pipe-based data sink
This is sample output - yours may be different.
-2

command >&-

lhunath · 2011-08-30 13:52:09 9
Show the single most recently modified file in a directory

List all files in a directory in reverse order by modified timestamp. When piped through tail the user will see the most recent file name.
This is sample output - yours may be different.
-2

ls -lFart |tail -n1

jambino · 2011-10-17 19:49:14 8
Print last modified time in 'date -- file' format
This is sample output - yours may be different.
-3

ls -alt /directory/ | awk '{ print $6 " " $7 " -- " $9 }'

gamefiend · 2011-02-17 12:50:38 6
Access variables inside a - piped while - loop

Consider the following simple situation [ reading something using while and read ] [See script 1 in sample output] --------------------------------------------------- The variable var is assigned with "nullll" at first. Inside the while loop [piped while] it is assigned with "whillleeee". [Onlly 2 assignments stmts]. Outside the loop the last assigned value for "var" [and no variable] inside the while can't be accessed [Due to pipe, var is executed in a sub shell]. In these type of situation variables can be accessed by modifying as follows. [See script 2 in sample output] ___________________________ Vary helpful when reading a set of items, say file names, stored on a file [or variable] to an array an use it later. Is there any other way 2 access variables inside and outside the loop ?? Show Sample Output
This is sample output - yours may be different.
```
script 1
__________________
#!/bin/bash

var="nullll"
echo "var=$var"
echo
str="aa
bb
cc"

echo "$str" | while read line
do
echo "line=$line"
echo "var=$var"
var="whillleeee"

done 

echo
echo "line=$line"
echo "var=$var"



------------------------------
Output
----------------------------
var=nullll

line=aa
var=nullll
line=bb
var=whillleeee
line=cc
var=whillleeee

var=nullll
___________________________________________________________
script 2---------------------------------------------------
___________________________________________________________

#!/bin/bash

var="nullll"
echo "var=$var"
echo
str="aa
bb
cc"

while read line
do
echo "line=$line"
echo "var=$var"
var="whillleeee"

done <<< "$str"

echo
echo "var=$var"

----------------------------------
Output
------------------------------------

var=nullll

line=aa
var=nullll
line=bb
var=whillleeee
line=cc
var=whillleeee

var=whillleeee
```
-5

while read line; do echo $line; done <<< "$var"

totti · 2011-09-22 16:53:32 6

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Check These Out

Create a mirror of a local folder, on a remote server

Create a exact mirror of the local folder "/root/files", on remote server 'remote_server' using SSH command (listening on port 22) (all files & folders on destination server/folder will be deleted)

A DESTRUCTIVE command to render a drive unbootable

THIS COMMAND IS DESTRUCTIVE. That said, lets assume you want to render your boot drive unbootable and reboot your machine. Maybe you want it to boot off the network and kickstart from a boot server for a fresh OS install. Replace /dev/fd0 with the device name of your boot drive and this DESTRUCTIVE command will render your drive unbootable. Your BIOS boot priority should be set to boot from HD first, then LAN.

Url Encode

mount an iso

mounts an ISO file to a directory on the target file system

Which processes are listening on a specific port (e.g. port 80)

swap out "80" for your port of interest. Can use port number or named ports e.g. "http"

nice disk usage, sorted by size, see description for full command

full command below, would not let me put full command in text box du -sk ./* | sort -nr | awk 'BEGIN{ pref[1]="K"; pref[2]="M"; pref[3]="G";} { total = total + $1; x = $1; y = 1; while( x > 1024 ) { x = (x + 1023)/1024; y++; } printf("%g%s\t%s\n",int(x*10)/10,pref[y],$2); } END { y = 1; while( total > 1024 ) { total = (total + 1023)/1024; y++; } printf("Total: %g%s\n",int(total*10)/10,pref[y]); }'

Convert CSV to JSON

Replace 'csv_file.csv' with your filename.

Rename files in batch

url shortner using google's shortner api

First get a api key for google url shortner from here https://developers.google.com/url-shortener/ Then replace the API_KEY in the command

Have a smile

Outputs utf-8 smileys

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

» all commands
» commands with 3 up-votes (commandlinefu3)
» commands with 10 up-votes (commandlinefu10)
» commands tagged pipe

Commands tagged pipe (23) the last day the last week the last month all time sorted by date votes

What's this?

Check These Out

Stay in the loop…

Commands tagged pipe (23)

sorted by