Commands tagged parsing (6)

  • url can be like any one of followings: url="MejbOFk7H6c" url="http://youtu.be/MejbOFk7H6c" url="https://youtube.com/watch?feature=player_embedded&v=MejbOFk7H6c#t" url="//www.youtube.com/v/MejbOFk7H6c?hl=ru_RU&version=3&rel=0" url="http://www.youtube.com/embed/MejbOFk7H6c?feature=player_embedded" If url mismatching, whole url will be returned. Show Sample Output


    3
    sh -c 'url="http://youtu.be/MejbOFk7H6c"; vid="`for i in ".*youtu\.be/\([^\/&?#]\+\)" ".*youtu.\+v[=/]\([^\/&?#]\+\)" ".*youtu.\+embed/\([^\/&?#]\+\)"; do expr "${url}" : "${i}"; done`"; if [ -n "${vid}" ]; then echo ${vid}; else echo "${url}"; fi'
    qwertyroot · 2013-09-04 19:33:09 8
  • You need to install WWW::Mechanize Perl module with # cpan -i WWW::Mezchanize or by searching mechanize | grep perl in your package manager With this command, you can get forms, images, headers too Show Sample Output


    2
    mech-dump --links --absolute http://www.commandlinefu.com
    sputnick · 2011-11-19 03:40:52 18
  • A really fun vim oneliner for auto documenting your option's parsing in your script. # print the text embeded in the case that parse options from command line. # the block is matched with the marker 'CommandParse' in comment, until 'esac' extract_cmdl_options() { # use vim for parsing: # 1st grep the case block and copy in register @p + unindent in the buffer of the file itself # 2nd filter lines which start with --opt or +opt and keep comment on hte following lines until an empty line # 3rd discard changes in the buffer and quit vim -n -es -c 'g/# CommandParse/+2,/^\s\+esac/-1 d p | % d | put p | % -c 'g/^\([-+]\+[^)]\+\))/,/^\(\s\+[^- \t#]\|^$\)/-1 p' \ -c 'q!' $0 } example code:http://snipplr.com/view/25059/display-embeded-comments-for-every-opt-usefull-for-auto-documenting-your-script/ Show Sample Output


    0
    vim -n -es -c 'g/# CommandParse/+2,/^\s\+esac/-1 d p | % d | put p | %<' -c 'g/^\([-+]\+[^)]\+\))/,/^\(\s\+[^- \t#]\|^$\)/-1 p' -c 'q!' $0
    syladmin · 2009-12-19 08:32:00 5
  • Sometimes, especially when parsing HTML, you want "all text between two tags, that doesn't contain another tag". For example, to grab only the contents of the innermost <div>s, something like: /<div\b[^>]*>((?:(?!<div).)*)</div>/ ...may be your best option to capture that text. It's not always needed, but is a powerful arrow in your regex quiver in those cases when you do need it. Note that, in general, regular expressions are the Wrong Choice for parsing HTML, anyway. Better approaches are solutions which let you navigate the HTML as a proper DOM. But sometimes, you just need to use the tools available to you. If you don't, then you have two problems.


    0
    Opening_tag((?:(?!Unwanted_tag).)*)Closing_tag
    DewiMorgan · 2012-02-28 02:54:57 3
  • # Usage: ftagmarks TAG BOOKMARKS.JSON ftagmarks Bash ~/.mozilla/firefox/*.default/bookmarkbackups/bookmarks-*.json Tag can be partial matching, e.g. input 'Bas' or 'ash' will match 'Bash' tag. # Exact tag matching: ftagmark(){ jq -r --arg t "$1" '.children[] as $i|if $i.root == "tagsFolder" then ([$i.children[] as $j|{title: ($j.title), urls: [$j.children[].uri]}]) else empty end|.[] as $k|if $k.title == $t then $k.urls else empty end|.[]?' "$2"; } Usage: ftagmark TAG BOOKMARKS.JSON # List all tags: ftagmarkl(){ jq -r '.children[] as $i | if $i.root == "tagsFolder" then $i.children[].title else empty end' "$1"; } Usage: ftagmarkl BOOKMARKS.JSON # Requires: `jq` - must have CLI JSON processor http://stedolan.github.io/jq Show Sample Output


    0
    ftagmarks(){ jq -r --arg t "$1" '.children[] as $i|if $i.root == "tagsFolder" then ([$i.children[] as $j|{title: ($j.title), urls: [$j.children[].uri]}]) else empty end|.[] as $k|if ($k.title|contains($t)) then $k.urls else empty end|.[]?' "$2"; }
    qwertyroot · 2016-12-24 15:12:04 15
  • Check out Gate number for your flight from CLI with Chrome, html2texgt and grep. Works on Arch Linux (Garuda) and probably will work on others. Requirements: * google chrome (might work with chromium as well) * installed html2text (on archlinux: sudo pacman -S python-html2text) * installed grep (comes by default with your OS) * the gate number should be visible at the given website (it's not existent too early before the flight and also disappears after the flight departed) Please don't forget to replace the link to appropriate one, matching your flight. You can also wrap this into something like `whlie true; do ...; sleep 60; done' and this will check and tell you the gate number maximum in 1 minute after it appears on Avinor website. Show Sample Output


    0
    google-chrome-stable --headless --dump-dom --disable-gpu "https://avinor.no/flight/?flightLegId=dy754-osl-trd-20220726&airport=OSL" 2>/dev/null | html2text | grep -A2 Gate
    sxiii · 2022-07-26 11:50:59 385

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands


Check These Out

Update all Docker Images

Which processes are listening on a specific port (e.g. port 80)
swap out "80" for your port of interest. Can use port number or named ports e.g. "http"

Multi-line grep
Using perl you can search for patterns spanning several lines, a thing that grep can't do. Append the list of files to above command or pipe a file through it, just as with regular grep. If you add the 's' modifier to the regex, the dot '.' also matches line endings, useful if you don't known how many lines you need are between parts of your pattern. Change '*' to '*?' to make it greedy, that is match only as few characters as possible. See also http://www.commandlinefu.com/commands/view/1764/display-a-block-of-text-with-awk to do a similar thing with awk. Edit: The undef has to be put in a begin-block, or a match in the first line would not be found.

Find the package that installed a command

Delete all but the latest 5 files, ignoring directories

cat stdout of multiple commands
Concatenate the stdout of multiple commands.

check open ports without netstat or lsof

Check command history, but avoid running it
!whatever will search your command history and execute the first command that matches 'whatever'. If you don't feel safe doing this put :p on the end to print without executing. Recommended when running as superuser.

Convert file type to unix utf-8
converts encoding of a file to unix utf-8 useful for data files that contain what would be usable ascii text but are encoded as mpeg or some other encoding that prevents you from doing common manipulations like 'sed'

check open ports without netstat or lsof


Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: