Commands using lftp sorted by votes

Commands using lftp (11)

sorted by

Mirror a directory structure from websites with an Apache-generated file indexes

wget/curl/friends are not good with mirroring files off websites, especially those with Apache-generated directory listings. These tools endlessly waste time downloading useless index HTML pages. lftp's mirror command does a better job without the mess.
This is sample output - yours may be different.
17

lftp -e "mirror -c" http://example.com/foobar/

tamasrepus · 2010-05-16 03:49:30 106
Backup sda5 partition to ftp ( using pipes and gziped backup )
This is sample output - yours may be different.
7

dd if=/dev/sda5 bs=2048 conv=noerror,sync | gzip -fc | lftp -u user,passwd domain.tld -e "put /dev/stdin -o backup-$(date +%Y%m%d%H%M).gz; quit"

sputnick · 2009-05-29 21:56:32 5
mirrors directory to a ftp server

http://lftp.yar.ru/
This is sample output - yours may be different.
7

lftp -ulogin,passwd -e "mirror --reverse /my/from/dir/ /ftp/target/dir/" ftp.server.xx

cp · 2011-04-01 12:40:11 6
Fastest segmented parallel sync of a remote directory over ssh

Mirror a remote directory using some tricks to maximize network speed. lftp:: coolest file transfer tool ever -u: username and password (pwd is merely a placeholder if you have ~/.ssh/id_rsa) -e: execute internal lftp commands set sftp:connect-program: use some specific command instead of plain ssh ssh:: -a -x -T: disable useless things -c arcfour: use the most efficient cipher specification -o Compression=no: disable compression to save CPU mirror: copy remote dir subtree to local dir -v: be verbose (cool progress bar and speed meter, one for each file in parallel) -c: continue interrupted file transfers if possible --loop: repeat mirror until no differences found --use-pget-n=3: transfer each file with 3 independent parallel TCP connections -P 2: transfer 2 files in parallel (totalling 6 TCP connections) sftp://remotehost:22: use sftp protocol on port 22 (you can give any other port if appropriate) You can play with values for --use-pget-n and/or -P to achieve maximum speed depending on the particular network. If the files are compressible removing "-o Compression=n" can be beneficial. Better create an alias for the command. Show Sample Output
This is sample output - yours may be different.
```
Transferring file `Maleficent.mkv'
`Maleficent.mkv', got 1890451456 of 9089412898 (20%) 6.82M/s eta:19m
ooooooo...............................ooooooo...............................ooooooo............

# the bar resembles the file being transferred, with 'o' representing data already received
```
4

lftp -u user,pwd -e "set sftp:connect-program 'ssh -a -x -T -c arcfour -o Compression=no'; mirror -v -c --loop --use-pget-n=3 -P 2 /remote/dir/ /local/dir/; quit" sftp://remotehost:22

colemar · 2014-10-17 00:29:34 10
Multi-segment file downloading with lftp

This is for files only, for directories 'mirror' has to be used.
This is sample output - yours may be different.
2

lftp -u user,pass ftp://site.com -e 'pget -c -n 6 file'

bugmenot · 2012-12-11 16:18:52 6
Gets directory and files tree listing from a FTP-server

Creates a file with contents like `du -a`, only it is remote server filesystem hierarchy. Very usefull then for grep-ing without remote connection.
This is sample output - yours may be different.
1

lftp -u<<credentials>> <<server>> -e "du -a;exit" > server-listing.txt

danbst · 2013-09-24 08:23:44 7
Multi-segment directory downloading with lftp

It works best if you first login and then do the fetch: lftp -u user,pass ftp://site.com/ mirror -c --parallel=3 --use-pget-n=5 "Some folder"
This is sample output - yours may be different.
0

lftp -u user,pass ftp://site.com/ -e 'mirror -c --parallel=3 --use-pget-n=5 "Some folder"'

bugmenot · 2012-11-24 11:29:48 15
Internet Speed Test

Make sure the file you use in your test is > 50mb to get good results. Dependancies: sudo apt-get install lftp iperf Show Sample Output
This is sample output - yours may be different.
```
lftp -e 'pget http://mirror.anl.gov/pub/centos/6.3/isos/x86_64/CentOS-6.3-x86_64-LiveCD.iso; exit; '
725621514 bytes transferred in 143 seconds (4.84M/s)  
```
0

lftp -e 'pget http://address_to_file; exit; '

o0110o · 2013-04-07 03:29:07 42
Use lftp to multi-threaded download files from websites
This is sample output - yours may be different.
0

lftp -c "pget -n 10 http://example.com/foo.bar"

specter · 2013-05-02 14:05:34 8
lftp backup from server to dev for a drupal site

sets the option set ftp:ssl-allow false, excludes some files I don't want to synchronize
This is sample output - yours may be different.
0

lftp -u user,pass -e "set ftp:ssl-allow false; mirror --exclude settings.php --exclude .htaccess serverdir devdir" serverhost

aufumy · 2015-09-08 21:25:41 10
Faciliate the work for lftp ('all' is needed if you wanna use it with getopts, otherwise its enough with the lftp line)

Add this to a script and you can do ./scriptname site help or ./scriptname mput etc etc.
This is sample output - yours may be different.
0

all="$(echo -e $*|awk '{for(i=3;i<=NF;++i)print $i}'|xargs)"; lftp -e open <HOSTNAME> -p <PORT> -u <USER>:<PASSWORD> -e "$all;exit"

wuseman1 · 2019-07-28 18:48:28 1362

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Check These Out

Replace all forward slashes with backward slashes

Use -i option to edit directly a file: sed -i 's|\/|\\|g' file

Never rewrites a file while copying (or moving)

Allows you to preserve your files when using cp, mv, ln, install or patch. When the target file exists, it will generate a file named XXX.~N~ (N is an auto-incremental number) instead of deleting the target file.

Quick and dirty RSS

runs an rss feed through sed replacing the closing tags with newlines and the opening tags with white space making it readable.

ROT13 whole file in vim.

gg puts the cursor at the begin g? ROT13 until the next mov G the EOF

Another Curl your IP command

Just another curl command to get your public facing IP

underscore to camelCase

Kills a process that is locking a file.

Useful when you're trying to unmount a volume and other sticky situations where a rogue process is annoying the hell out of you.

Buffer in order to avoir mistakes with redirections that empty your files

A common mistake in Bash is to write command-line where there's command a reading a file and whose result is redirected to that file. It can be easily avoided because of : 1) warnings "-bash: file.txt: cannot overwrite existing file" 2) options (often "-i") that let the command directly modify the file but I like to have that small function that does the trick by waiting for the first command to end before trying to write into the file. Lots of things could probably done in a better way, if you know one...

Quickly build ulimit command from current values

It is helpful to know the current limits placed on your account, and using this shortcut is a quick way to figuring out which values to change for optimization or security. Alias is: $ alias ulimith="command ulimit -a|sed 's/^.*$[a-z]$)$.*$$/-\1\2/;s/^/ulimit /'|tr '\n' ' ';echo" Here's the result of this command: $ ulimit -c 0 -d unlimited -e 0 -f unlimited -i 155648 -l 32 -m unlimited -n 8192 -p 8 -q 819200 -r 0 -s 10240 -t unlimited -u unlimited -v unlimited -x unlimited $ ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 155648 max locked memory (kbytes, -l) 32 max memory size (kbytes, -m) unlimited open files (-n) 8192 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 10240 cpu time (seconds, -t) unlimited max user processes (-u) unlimited virtual memory (kbytes, -v) unlimited file locks (-x) unlimited

Happy New Year!

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

» all commands
» commands with 3 up-votes (commandlinefu3)
» commands with 10 up-votes (commandlinefu10)
» commands using lftp

Commands using lftp (11) the last day the last week the last month all time sorted by date votes

What's this?

Check These Out

Stay in the loop…

Commands using lftp (11)

sorted by