Commands tagged encoding (12)

  • This is just a proof of concept: A FILE WHICH CAN AUTOMOUNT ITSELF through a SIMPLY ENCODED script. It takes advantage of the OFFSET option of mount, and uses it as a password (see that 9191? just change it to something similar, around 9k). It works fine, mounts, gets modified, updated, and can be moved by just copying it. USAGE: SEE SAMPLE OUTPUT The file is composed of three parts: a) The legible script (about 242 bytes) b) A random text fill to reach the OFFSET size (equals PASSWORD minus 242) c) The actual filesystem Logically, (a)+(b) = PASSWORD, that means OFFSET, and mount uses that option. PLEASE NOTE: THIS IS NOT AN ENCRYPTED FILESYSTEM. To improve it, it can be mounted with a better encryption script and used with encfs or cryptfs. The idea was just to test the concept... with one line :) It applies the original idea of http://www.commandlinefu.com/commands/view/7382/command-for-john-cons for encrypting the file. The embedded bash script can be grown, of course, and the offset recalculation goes fine. I have my own version with bash --init-file to startup a bashrc with a well-defined environment, aliases, variables. Show Sample Output


    6
    dd if=/dev/zero of=T bs=1024 count=10240;mkfs.ext3 -q T;E=$(echo 'read O;mount -o loop,offset=$O F /mnt;'|base64|tr -d '\n');echo "E=\$(echo $E|base64 -d);eval \$E;exit;">F;cat <(dd if=/dev/zero bs=$(echo 9191-$(stat -c%s F)|bc) count=1) <(cat T;rm T)>>F
    rodolfoap · 2013-01-31 01:38:30 5

  • 4
    iconv -f utf8 -t utf16 /path/to/file
    imsaar · 2009-12-01 21:02:58 0
  • Useful if non-ascii characters in filenames have been improperly encoded. Replace "PROBLEM" with the incorrect characters (e.g. 'é'), and "FIX" with the correct ones (e.g. '?').


    3
    for i in *;do mv "$i" "$(echo $i | sed s/PROBLEM/FIX/g)";done
    AlecSchueler · 2009-06-28 01:50:25 4
  • Get gzip compressed web page using wget. Caution: The command will fail in case website doesn't return gzip encoded content, though most of thw websites have gzip support now a days.


    3
    wget -q -O- --header\="Accept-Encoding: gzip" <url> | gunzip > out.html
    ashish_0x90 · 2010-11-27 22:14:42 0

  • 3
    find . -iname *.java -type f -exec bash -c "iconv -f WINDOWS-1252 -t UTF-8 {} > {}.tmp " \; -exec mv {}.tmp {} \;
    miguelbaldi · 2012-01-12 20:00:26 0
  • This command gives you the charset of a text file, which would be handy if you have no idea of the encoding. Show Sample Output


    2
    file -i <textfile>
    juvenn · 2009-09-08 01:33:19 1
  • http://www.mplayerhq.hu/DOCS/HTML/en/menc-feat-mpeg.html MEncoder can create MPEG (MPEG-PS) format output files. Usually, when you are using MPEG-1 or MPEG-2 video, it is because you are encoding for a constrained format such as SVCD, VCD, or DVD. To change MEncoder's output file format, use the -of mpeg option. Creating an MPEG-1 file suitable to be played on systems with minimal multimedia support, such as default Windows installs: mencoder input.avi -of mpeg -mpegopts format=mpeg1:tsaf:muxrate=2000 \ -o output.mpg -oac lavc -lavcopts acodec=mp2:abitrate=224 -ovc lavc \ -lavcopts vcodec=mpeg1video:vbitrate=1152:keyint=15:mbd=2:aspect=4/3


    1
    mencoder input.avi -of mpeg -ovc lavc -lavcopts vcodec=mpeg1video \ -oac copy other_options -o output.mpg
    slishan · 2010-01-18 13:12:03 0
  • Really helpfull when play with files having spaces an other bad name. Easy to store and access names and path in just a field while saving it in a file. This format (URL) is directly supported by nautilus and firefox (and other browsers) Show Sample Output


    1
    convert_path2uri () { echo -n 'file://'; echo -n "$1" | perl -pe 's/([^a-zA-Z0-9_\/.])/sprintf("%%%.2x", ord($1))/eg' ;} #convert2uri '/tmp/a b' ### convert file path to URI
    totti · 2013-07-01 08:54:45 0
  • Like the original command, but the -f allows this one to succeed even if the website returns uncompressed data. From gzip(1) on the -f flag: If the input data is not in a format recognized by gzip, and if the --stdout is also given, copy the input data without change to the standard output: let zcat behave as cat.


    1
    wget -q -O- --header="Accept-Encoding: gzip" <url> | gzip -cdf > out.html
    tempusername · 2014-11-29 20:42:21 0

  • 0
    uchardet <filename>
    fltman · 2014-11-28 07:47:13 0
  • I still don't know why mutt (or offlineimap or whatever in between), is borking the encoding of my files, but this fixes it. Show Sample Output


    0
    fix_saved_files() { for i in "[email protected]" ; do mv $i $(php -r '$or = "'$i'"; mb_internal_encoding("UTF-8"); echo mb_decode_mimeheader($or) . "\n";') ; done; }
    guillaume · 2014-12-27 20:37:16 0
  • Also works with files: cat file Hello world base64 file SGVsbG8gd29ybGQK To decrypt use the -d option: echo SGVsbG8gd29ybGQK | base64 -d Hello world Show Sample Output


    -3
    echo "Hello world" | base64
    P17 · 2009-04-27 15:00:08 8

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands


Check These Out

Remove BOM (Byte Order Mark) from text file
Takes file (text.txt), removes BOM from it, and outputs the result to a new file (newFile.txt). BOM is "Byte Order Mark" ([http://en.wikipedia.org/wiki/Byte_order_mark]), an invisible, non-breaking, zero-length character. In other words, if you see a DIFF with "" at the beginning, you've got a byte order mark, which can't be removed without this command or a hex editor. It can appear for a number of reasons, such as getting copied to/from a UNIX filesystem...

Retry the previous command until it exits successfully

Install pip with Proxy
Installs pip packages defining a proxy

m4a to mp3 conversion with ffmpeg and lame
A batch file version of the same command would be: for f in *.m4a; do ffmpeg -i "$f" -acodec libmp3lame -ab 256k "${f%.m4a}.mp3"; done

Split and join with split and cat.
`split -b 1k file` splits files into 1k chunks. Rejoin them with `cat x* > file`.

Find file containing namespace in a directory of jar files.
You could subsitute javax.servlet for any namespace you need.

regex for turning a URL into a real hyperlink (i.e. for posting somewhere that accepts basic html)
This should work with anything://url.whatever etc etc ;)

Kill all processes that listen to ports begin with 50 (50, 50x, 50xxx,...)
Run netstat as root (via sudo) to get the ID of the process listening on the desired socket. Use awk to 1) match the entry that is the listening socket, 2) matching the exact port (bounded by leading colon and end of column), 3) remove the trailing slash and process name from the last column, and finally 4) use the system(…) command to call kill to terminate the process. Two direct commands, netstat & awk, and one forked call to kill. This does kill the specific port instead of any port that starts with 50. I consider this to be safer.

Link a deep tree of files all into on directory
If you want to pull all of the files from a tree that has mixed files and directories containing files, this will link them all into a single directory. Beware of filesystem files-per-directory limits.

Print just line 4 from a textfile
Prints the 4th line and then quits. (Credit goes to flatcap in comments: http://www.commandlinefu.com/commands/view/6031/print-just-line-4-from-a-textfile#comment.)


Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: