Commands tagged json (43)

  • You can use a site like http://www.jsonlint.com/ or use the command line to validate your long and complex json data. This is part of the simplejson package for python http://undefined.org/python/#simplejson. Wrong json expression example: echo '{ 1.2:3.4}' | python -m simplejson.tool Expecting property name: line 1 column 2 (char 2) Show Sample Output


    14
    echo '{"json":"obj"}' | python -m simplejson.tool
    azeey · 2009-05-10 13:58:05 4
  • Validates and pretty-prints the content fetched from the URL. Show Sample Output


    12
    curl -s "http://feeds.delicious.com/v2/json?count=5" | python -m json.tool | less -R
    keimlink · 2010-03-24 09:15:12 3

  • 11
    ruby -ryaml -rjson -e 'puts YAML.dump(JSON.parse(STDIN.read))' < file.json > file.yaml
    wejn · 2013-04-24 07:20:37 0
  • Convert JSON to YAML. Note that you'll need to have PyYaml installed.


    9
    python -c 'import sys, yaml, json; yaml.safe_dump(json.load(sys.stdin), sys.stdout, default_flow_style=False)' < file.json > file.yaml
    tebeka · 2013-04-24 00:31:39 2
  • Leave out pygmentize or `pip install pygments` first. Show Sample Output


    5
    alias pp='python -mjson.tool|pygmentize -l js'
    wires · 2014-05-22 10:29:44 3
  • Converts YAML file to JSON. Note that you'll need to install PyYAML. Also some YAML data types (like dates) are not supported by JSON).


    4
    python -c 'import sys, yaml, json; json.dump(yaml.load(sys.stdin), sys.stdout, indent=4)' < file.yaml > file.json
    tebeka · 2013-04-24 00:28:55 1
  • Replace 'csv_file.csv' with your filename.


    4
    python -c "import csv,json;print json.dumps(list(csv.reader(open('csv_file.csv'))))"
    malathion · 2017-05-17 19:35:53 1
  • The FLAC audio must be encoded at 16000Hz sampling rate (SoX is your friend). Outputs a short JSON string, the actual speech is in the hypotheses->utterance, the accuracy is stored in hypotheses->confidence (ranging from 0 to 1). Google also accepts audio in some special speex format (audio/x-speex-with-header-byte), which is much smaller in comparison with losless FLAC, but I haven't been able to encode such a sample. Show Sample Output


    3
    wget -q -U "Mozilla/5.0" --post-file speech.flac --header="Content-Type: audio/x-flac; rate=16000" -O - "http://www.google.com/speech-api/v1/recognize?lang=en-us&client=chromium"
    sairon · 2011-03-08 13:39:01 0
  • jq is amazing for manipulating json on the commandline, but the developers have some weird ideas about how to handle shell redirections. This command works around them. Further reading: https://github.com/stedolan/jq/issues/1110


    3
    diff <(jq . -M -S < old.json) <(jq . -M -S < new.json)
    malathion · 2018-10-11 20:59:48 0
  • Uses pygmentize and python to create indented and colorized JSON output Show Sample Output


    2
    alias pp='python -mjson.tool | pygmentize -l javascript'
    wires · 2012-10-16 13:55:38 1

  • 2
    curl -s httpbin.org/ip | jq -r .origin
    tebeka · 2016-08-11 04:20:48 1
  • * Output is jq compatible * Output is single lines - unix compatible * Multiple files supported


    2
    perl -MYAML::XS=LoadFile -MJSON::XS=encode_json -e 'for (@ARGV) { for (LoadFile($_)) { print encode_json($_),"\n" } }' *.yaml
    bsb · 2017-11-17 09:33:56 0
  • With this command you can convert a tab separate file (TSV) into a JSON file with jq. For example, this input.tsv i-0b9adca882e5e6326 172.16.0.188 i-088dd69e5c3624888 172.16.0.102 i-0e70eac180537d4aa 172.16.0.85 will produce the showed output. Show Sample Output


    2
    cat input.tsv | jq --raw-input --slurp 'split("\n") | map(split("\t")) | .[0:-1] | map( { "id": .[0], "ip": .[1] } )'
    nordri · 2019-10-01 10:52:35 3
  • Using the csv tool `miller` you can transform a csv file into a json array of objects, where the properties are the values of the csv header line and the values are the values of the subsequent lines. Show Sample Output


    2
    mlr --c2j --jlistwrap cat file.csv
    pepa65 · 2019-12-06 07:16:58 7
  • wget -qO - "http://www.google.com/dictionary/json?callback=dict_api.callbacks.id100&q=steering+wheel&sl=en&tl=en&restrict=pr,de&client=te" this does the actual google dictionary query, returns a JSON string encapsulated in some fancy tag sed 's/dict_api\.callbacks.id100.//' here we remove the tag beginning sed 's/,200,null)//' and here the tag end There are also some special characters which could cause problems with some JSON parsers, so if you get some errors, this is probably the case (sed is your friend). I laso like to trim the "webDefinitions" part, because it (sometimes) contains misleading information. sed 's/\,\"webDefinitions.*//' (but remember to append a "}" at the end, because the JSON string will be invalid) The output also contains links to mp3 files with pronounciation. As of now, this is only usable in the English language. If you choose other than English, you will only get webDefinitions (which are crap).


    1
    wget -qO - "http://www.google.com/dictionary/json?callback=dict_api.callbacks.id100&q=steering+wheel&sl=en&tl=en&restrict=pr,de&client=te" | sed 's/dict_api\.callbacks.id100.//' | sed 's/,200,null)//'
    sairon · 2011-03-08 15:00:39 0
  • For situations where you keep JSON in a VCS and you want your diffs to be sane, such as within a Chef configuration repo.


    1
    cat foo.json | python -mjson.tool
    jeffgage · 2013-05-06 20:11:05 0
  • You need to install the Catmandu Perl module via `cpanm Catmandu`


    1
    catmandu convert JSON to YAML < file.json > file.yaml
    unixmonkey73469 · 2014-02-25 08:27:05 0
  • This is based on __unixmonkey73469__ answer. You will need to supply `--multiline 1` option to JSON importer if your .json is multiline (i.e. it was prettyfied) And you still need catmandu installed via `cpanm Catmandu`


    1
    catmandu convert JSON --multiline 1 to YAML < file.json > file.yaml
    pavelkaroukin · 2014-09-29 16:45:24 0
  • If you tried the other Python version of Convert JSON to YAML and you end up with lines that has "!!python/unicode", this version of the command is for you.


    1
    python -c 'import sys, yaml, json; yaml.safe_dump(json.load(sys.stdin), sys.stdout, allow_unicode=True)' < foo.json > foo.yaml
    renoirb · 2015-01-20 15:58:29 0
  • Requires installing json2yaml via npm: npm install -g json2yaml (can also pipe from stdin) Ref: https://www.npmjs.com/package/json2yaml


    1
    json2yaml ./example.json > ./example.yml
    quaddo · 2017-06-13 22:25:08 0
  • Based / Inspired by malathion's below command http://www.commandlinefu.com/commands/view/20528/convert-csv-to-json Is written for python3 and is very easy to use csv2json *csv will convert all files ending in csv to json eg csv2json file.csv will output a file to file.json Validity of json tested in python3 and below site https://jsonformatter.curiousconcept.com/


    1
    csv2json() { for file in $@; do python -c "import csv,json,fileinput; print(json.dumps(list(csv.reader(fileinput.input()))))" "$file" 1> "${file%%csv}json"; done; }
    snipertyler · 2017-06-28 04:28:32 0
  • the `jq` tool can also be used do validate json files and pretty print output `cat file.json | jq` available on several platforms, including newer debian-based systems via `#sudo apt install jq`, mac via `brew install jq`, and from source https://stedolan.github.io/jq/download/ Show Sample Output


    1
    cat file.json | jq
    daemeon · 2018-10-01 23:05:27 3
  • The `jq` tool can also be used do validate json files and pretty print output: ` jq < file.json` Available on several platforms, including newer debian-based systems via `#sudo apt install jq`, mac via `brew install jq`, and from source https://stedolan.github.io/jq/download/ This alternative to the original avoids the useless use of cat Show Sample Output


    1
    jq < file.json
    malathion · 2018-10-03 18:46:46 10

  • 0
    curl -s "http://services.digg.com/stories?link=$NEWSURL&appkey=http://www.whatever.com&type=json" | python -m simplejson.tool | grep diggs
    jyro · 2009-08-24 08:49:20 0
  • If you want all the URLs from all the sessions, you can use : perl -lne 'print for /url":"\K[^"]+/g' ~/.mozilla/firefox/*/sessionstore.js Thanks to tybalt89 ( idea of the "for" statement ). For perl purists, there's JSON and File::Slurp modules, buts that's not installed by default.


    0
    perl -lne 'print for /url":"\K[^"]+/g' $(ls -t ~/.mozilla/firefox/*/sessionstore.js | sed q)
    sputnick · 2009-12-14 00:51:54 0
  •  1 2 > 

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands


Check These Out

Find which service was used by which port number

Create the oauth token required for a Twitter stream feed
This is the THIRD in a set of five commands. See my other commands for the previous two. This step creates the oauth 1.0 token as explained in http://oauth.net/core/1.0/ The token is required for a Twitter filtered stream feed (and almost all Twitter API calls) This token is simply an encrypted version of your base string. The encryption key used is your hmac. The last part of the command scans the Base64 token string for '+', '/', and '=' characters and converts them to percentage-hex escape codes. (URI-escapeing). This is also a good example of where the $() syntax of Bash command substitution fails, while the backtick form ` works - the right parenthesis in the case statement causes a syntax error if you try to use the $() syntax here. See my previous two commands step1 and step2 to see how the base string variable $b and hmac variable $hmac are generated.

Get http headers for an url

List just the executable files (or directories) in current directory
A bit shorter ;)

Set random background image in gnome
Changes your desktop background image in gnome. Update the directory to wherever you keep your wallpapers. I like to create a sub-directory in my Wallpaper folder called "cycle" that I use to define the wallpapers I wish to loop in cron. ex: $ gconftool-2 -t str -s /desktop/gnome/background/picture_filename "$(find ~/Wallpapers/cycle -type f | shuf -n1)"

Display network pc "name" and "workgroup"
Checks for PC samba name and workgroup. Works fine for Windows hosts and Linux/UNIX PCs running Samba.

Use Vim to convert text to HTML.
``vimhtml somefile.txt`` will open vim for the HTML convertion and close it immediately after its done, leaving you with somefile.html which you can later use in your website or whatever.

wrap long lines of a text
wraps text lines at the specified width (90 here). -s option is to force to wrap on blank characters -b count bytes instead of columns

Use the builtin ':' bash command to increment variables
I just found another use for the builtin ':' bash command. It increments counters for me in a loop if a certain condition is met... : [arguments] No effect; the command does nothing beyond expanding arguments and performing any specified redirections. A zero exit code is returned.

Benchmark SQL Query
Benchmark a SQL query against MySQL Server. The example runs the query 10 times, and you get the average runtime in the output. To ensure that the query does not get cached, use `RESET QUERY CACHE;` on top in the query file.


Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: