Commands tagged tsv (5)

  • With this command you can convert a tab separate file (TSV) into a JSON file with jq. For example, this input.tsv i-0b9adca882e5e6326 172.16.0.188 i-088dd69e5c3624888 172.16.0.102 i-0e70eac180537d4aa 172.16.0.85 will produce the showed output. Show Sample Output


    2
    cat input.tsv | jq --raw-input --slurp 'split("\n") | map(split("\t")) | .[0:-1] | map( { "id": .[0], "ip": .[1] } )'
    nordri · 2019-10-01 10:52:35 3
  • Frustrated with the manual domain migration process AWS has, I unsuccessfully tried to install cli53, route53-transfer. I instead wrote this oneliner to ease the export (which is not supported via the AWS console ATM). The output can be easily pasted into the "Import Hosted Zone" dialog in Route53. SOA/NS records are excluded since they cannot be automatically imported. Show Sample Output


    1
    echo -e "\$ORIGIN\tumccr.org.\n\$TTL\t1h\n" && aws route53 list-resource-record-sets --hosted-zone-id Z1EEXAMPLE9SF3 | jq -r '.ResourceRecordSets[] | [.Name, .Type, .ResourceRecords[0].Value] | join("\t")' - | grep -vE "NS|SOA"
    brainstorm · 2018-06-15 01:19:21 0
  • Will split the std input lines into files grouped by the 5th column content.


    0
    awk -F'\t' '{print $0 >>$5.tsv}'
    pykler · 2012-05-16 18:18:16 0
  • Use sed to search and replace pipes for tabs in file stream with backup


    0
    sed -i.bak -e s/\|/\\t/g filepath.tsv
    johnmoxon · 2014-11-04 00:14:59 0
  • Grabs the first JSON file in the directory, reads its keys, prints TSV, then prints all the json files' values as TSV. Nested objects appear as json. Unhappy times if your json has literal tabs in it. Show Sample Output


    0
    jq -r 'keys | join("\t")' $(ls -f *.json | head -1) && jq -Sr 'to_entries | [ .[] | .value | tostring ] | join("\t")' *.json
    drjeats · 2016-04-08 23:30:30 0

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands


Check These Out

Save an HTML page, and covert it to a .pdf file
Uses htmldoc to perform the conversion

write text or append to a file
If you just want to write or append some text to a file without having to run a text editor, run this command. After running the command, start typing away. To exit, type . on a line by itself. Replacing the >> with a single > will let you overwrite your file.

Buffer in order to avoir mistakes with redirections that empty your files
A common mistake in Bash is to write command-line where there's command a reading a file and whose result is redirected to that file. It can be easily avoided because of : 1) warnings "-bash: file.txt: cannot overwrite existing file" 2) options (often "-i") that let the command directly modify the file but I like to have that small function that does the trick by waiting for the first command to end before trying to write into the file. Lots of things could probably done in a better way, if you know one...

Change timestamp on a file
-a for access time, -m for modification time, -c do not create any files, -t timestamp

Get Futurama quotations from slashdot.org servers

Expand shortened URLs
curl(1) is more portable than wget(1) across Unices, so here is an alternative doing the same thing with greater portability. This shell function uses curl(1) to show what site a shortened URL is pointing to, even if there are many nested shortened URLs. This is a great way to test whether or not the shortened URL is sending you to a malicious site, or somewhere nasty that you don't want to visit. The sample output is from: $ expandurl http://t.co/LDWqmtDM

foo <--> german translation with dict.leo.org
Translate strings from non-german to german (and vice versa) using LEO. Put it in your ~/.bashrc. Usage: $ leo words   To use another language other than english, use an option: $ leo -xx words Valid language options: ch - chinese en - english es - spanish fr - french it - italian pl - polish pt - portuguese ru - russian The other language will always be german!

Which processes are listening on a specific port (e.g. port 80)
swap out "80" for your port of interest. Can use port number or named ports e.g. "http"

Sniff ONLY POP3 authentication by intercepting the USER command
dsniff is general purpose password sniffer, it handles *lots* of different protocols, but it also handles tcp-style expressions for limiting analyzed traffic - so I can limit it to work on pop3 only.

Quickly graph a list of numbers
Useful when you've produced a large file of numbers, and want to quickly see the distribution. The value of y halfway along the x axis is the median. Simple! Just create the listOfNumbers.txt file with a number on each line to try it out.


Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: