Commands tagged tsv (5)

  • With this command you can convert a tab separate file (TSV) into a JSON file with jq. For example, this input.tsv i-0b9adca882e5e6326 172.16.0.188 i-088dd69e5c3624888 172.16.0.102 i-0e70eac180537d4aa 172.16.0.85 will produce the showed output. Show Sample Output


    2
    cat input.tsv | jq --raw-input --slurp 'split("\n") | map(split("\t")) | .[0:-1] | map( { "id": .[0], "ip": .[1] } )'
    nordri · 2019-10-01 10:52:35 1
  • Frustrated with the manual domain migration process AWS has, I unsuccessfully tried to install cli53, route53-transfer. I instead wrote this oneliner to ease the export (which is not supported via the AWS console ATM). The output can be easily pasted into the "Import Hosted Zone" dialog in Route53. SOA/NS records are excluded since they cannot be automatically imported. Show Sample Output


    1
    echo -e "\$ORIGIN\tumccr.org.\n\$TTL\t1h\n" && aws route53 list-resource-record-sets --hosted-zone-id Z1EEXAMPLE9SF3 | jq -r '.ResourceRecordSets[] | [.Name, .Type, .ResourceRecords[0].Value] | join("\t")' - | grep -vE "NS|SOA"
    brainstorm · 2018-06-15 01:19:21 0
  • Will split the std input lines into files grouped by the 5th column content.


    0
    awk -F'\t' '{print $0 >>$5.tsv}'
    pykler · 2012-05-16 18:18:16 0
  • Use sed to search and replace pipes for tabs in file stream with backup


    0
    sed -i.bak -e s/\|/\\t/g filepath.tsv
    johnmoxon · 2014-11-04 00:14:59 0
  • Grabs the first JSON file in the directory, reads its keys, prints TSV, then prints all the json files' values as TSV. Nested objects appear as json. Unhappy times if your json has literal tabs in it. Show Sample Output


    0
    jq -r 'keys | join("\t")' $(ls -f *.json | head -1) && jq -Sr 'to_entries | [ .[] | .value | tostring ] | join("\t")' *.json
    drjeats · 2016-04-08 23:30:30 0

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands


Check These Out

uniq without pre-sorting
Reads stdin, and outputs each line only once - without sorting ahead of time. This does use more memory than your system's sort utility.

set your ssd disk as a non-rotating medium
if you still get a permissions error using sudo, then nano the file: sudo nano -w /sys/block/sdb/queue/rotational and change 1 to 0 this thread: http://www.ocztechnologyforum.com/forum/showpost.php?p=369836&postcount=15 says that this will "help the block layer to optimize a few decisions"

tar the current directory wihtout the absolute path
tars the current directory (and its children) in an archive of the same name (plus ".tar" :)) in the parent directory without the absolute path, so that when the archive is extracted, only the current directory name is created for the path. Assumes bash/zsh.

Which processes are listening on a specific port (e.g. port 80)
swap out "80" for your port of interest. Can use port number or named ports e.g. "http"

Apache CLF access log format to CSV converter
- excel date compatible with a separate hour field - added a fixed 1 for easier request counter aggregation - split URL in directory, filename, fileext, query - used with tomcat valve with response bytes replaced by elapsed time

Dump a web page
Useful to browse dangerous web sites.

Reload all sysctl variables without reboot
Use `sysctl -p` without argument will only load /etc/sysctl.conf, but user configs always put in /etc/sysctl.d/*.conf, `sysctl --system` will load all the config files

Convert one file from ISO-8859-1 to UTF-8.
Nothing fancy it just converts one file from one character encoding into another one.

posts an xml file to a webservice with curl

Remove invalid key from the known_hosts file for the IP address of a host
Quick shortcut if you know the hostname and want to save yourself one step for looking up the IP address separately.


Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: