Commands tagged parsing (6)

  • url can be like any one of followings: url="MejbOFk7H6c" url="http://youtu.be/MejbOFk7H6c" url="https://youtube.com/watch?feature=player_embedded&v=MejbOFk7H6c#t" url="//www.youtube.com/v/MejbOFk7H6c?hl=ru_RU&version=3&rel=0" url="http://www.youtube.com/embed/MejbOFk7H6c?feature=player_embedded" If url mismatching, whole url will be returned. Show Sample Output


    3
    sh -c 'url="http://youtu.be/MejbOFk7H6c"; vid="`for i in ".*youtu\.be/\([^\/&?#]\+\)" ".*youtu.\+v[=/]\([^\/&?#]\+\)" ".*youtu.\+embed/\([^\/&?#]\+\)"; do expr "${url}" : "${i}"; done`"; if [ -n "${vid}" ]; then echo ${vid}; else echo "${url}"; fi'
    qwertyroot · 2013-09-04 19:33:09 9
  • You need to install WWW::Mechanize Perl module with # cpan -i WWW::Mezchanize or by searching mechanize | grep perl in your package manager With this command, you can get forms, images, headers too Show Sample Output


    2
    mech-dump --links --absolute http://www.commandlinefu.com
    sputnick · 2011-11-19 03:40:52 18
  • A really fun vim oneliner for auto documenting your option's parsing in your script. # print the text embeded in the case that parse options from command line. # the block is matched with the marker 'CommandParse' in comment, until 'esac' extract_cmdl_options() { # use vim for parsing: # 1st grep the case block and copy in register @p + unindent in the buffer of the file itself # 2nd filter lines which start with --opt or +opt and keep comment on hte following lines until an empty line # 3rd discard changes in the buffer and quit vim -n -es -c 'g/# CommandParse/+2,/^\s\+esac/-1 d p | % d | put p | % -c 'g/^\([-+]\+[^)]\+\))/,/^\(\s\+[^- \t#]\|^$\)/-1 p' \ -c 'q!' $0 } example code:http://snipplr.com/view/25059/display-embeded-comments-for-every-opt-usefull-for-auto-documenting-your-script/ Show Sample Output


    0
    vim -n -es -c 'g/# CommandParse/+2,/^\s\+esac/-1 d p | % d | put p | %<' -c 'g/^\([-+]\+[^)]\+\))/,/^\(\s\+[^- \t#]\|^$\)/-1 p' -c 'q!' $0
    syladmin · 2009-12-19 08:32:00 7
  • Sometimes, especially when parsing HTML, you want "all text between two tags, that doesn't contain another tag". For example, to grab only the contents of the innermost <div>s, something like: /<div\b[^>]*>((?:(?!<div).)*)</div>/ ...may be your best option to capture that text. It's not always needed, but is a powerful arrow in your regex quiver in those cases when you do need it. Note that, in general, regular expressions are the Wrong Choice for parsing HTML, anyway. Better approaches are solutions which let you navigate the HTML as a proper DOM. But sometimes, you just need to use the tools available to you. If you don't, then you have two problems.


    0
    Opening_tag((?:(?!Unwanted_tag).)*)Closing_tag
    DewiMorgan · 2012-02-28 02:54:57 4
  • # Usage: ftagmarks TAG BOOKMARKS.JSON ftagmarks Bash ~/.mozilla/firefox/*.default/bookmarkbackups/bookmarks-*.json Tag can be partial matching, e.g. input 'Bas' or 'ash' will match 'Bash' tag. # Exact tag matching: ftagmark(){ jq -r --arg t "$1" '.children[] as $i|if $i.root == "tagsFolder" then ([$i.children[] as $j|{title: ($j.title), urls: [$j.children[].uri]}]) else empty end|.[] as $k|if $k.title == $t then $k.urls else empty end|.[]?' "$2"; } Usage: ftagmark TAG BOOKMARKS.JSON # List all tags: ftagmarkl(){ jq -r '.children[] as $i | if $i.root == "tagsFolder" then $i.children[].title else empty end' "$1"; } Usage: ftagmarkl BOOKMARKS.JSON # Requires: `jq` - must have CLI JSON processor http://stedolan.github.io/jq Show Sample Output


    0
    ftagmarks(){ jq -r --arg t "$1" '.children[] as $i|if $i.root == "tagsFolder" then ([$i.children[] as $j|{title: ($j.title), urls: [$j.children[].uri]}]) else empty end|.[] as $k|if ($k.title|contains($t)) then $k.urls else empty end|.[]?' "$2"; }
    qwertyroot · 2016-12-24 15:12:04 15
  • Check out Gate number for your flight from CLI with Chrome, html2texgt and grep. Works on Arch Linux (Garuda) and probably will work on others. Requirements: * google chrome (might work with chromium as well) * installed html2text (on archlinux: sudo pacman -S python-html2text) * installed grep (comes by default with your OS) * the gate number should be visible at the given website (it's not existent too early before the flight and also disappears after the flight departed) Please don't forget to replace the link to appropriate one, matching your flight. You can also wrap this into something like `whlie true; do ...; sleep 60; done' and this will check and tell you the gate number maximum in 1 minute after it appears on Avinor website. Show Sample Output


    0
    google-chrome-stable --headless --dump-dom --disable-gpu "https://avinor.no/flight/?flightLegId=dy754-osl-trd-20220726&airport=OSL" 2>/dev/null | html2text | grep -A2 Gate
    sxiii · 2022-07-26 11:50:59 548

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands


Check These Out

Processes by CPU usage

Stream YouTube URL directly to mplayer.

Create named LUKS encrypted volume
You need to be root to do this. So check the command before running it. You enter the same password for Enter LUKS passphrase: Verify passphrase: Enter passphrase for /dev/loopn: ___ You can then copy the .img file to somewhere else. Loop it it with losetup -f IMAGENAME.img and then mount it with a file manager (eg nemo) or run mount /dev/loopn /media/mountfolder Acts similar to a mounted flash drive

Auto Rotate Cube (compiz)
Strip my code to: wmctrl -o 0,0 # autorotates to the first face. In fact [0-1279],0 wmctrl - 1280,0 # goes to the second face wmctrl -o 2560,0 # goes to the third face, and so on. # Use multiples of the horizontal display resolution. My example work for 1280x800 display, been 1280 the number of interest. Tweak the number, try a biiiig one and see your cube spinning... I put a complex example to show how fun things can be, even for my ademco and paradox alarm central network advisor interface xpto etc. It rotates two faces, print the alarm message, and goes back tho where it was. Tested on BIGLINUX 4.2, equivalent to ubuntu LTS hardy. Do not forget to activate 3D efects ( compiz cube )

Show all available colors on your terminal.
Using perl and tput, show all the colors with numbers that your actual $TERM can handle. If want to remove the numbers at beginning of new line, it should be something like this: $perl -E 'say `tput setb $_`," "x `tput cols`, `tput sgr0` for 0 .. (`tput colors` - 1)'

Get AWS temporary credentials ready to export based on a MFA virtual appliance
You might want to secure your AWS operations requiring to use a MFA token. But then to use API or tools, you need to pass credentials generated with a MFA token. This commands asks you for the MFA code and retrieves these credentials using AWS Cli. To print the exports, you can use: `awk '{ print "export AWS_ACCESS_KEY_ID=\"" $1 "\"\n" "export AWS_SECRET_ACCESS_KEY=\"" $2 "\"\n" "export AWS_SESSION_TOKEN=\"" $3 "\"" }'` You must adapt the command line to include: * $MFA_IDis ARN of the virtual MFA or serial number of the physical one * TTL for the credentials

Make info pages much less painful
The pinfo package makes info pages much more bearable. It is a ncurses-based POSIX utility for viewing info and man pages using lynx style keyboard shortcuts and rendering. Links are highlighted blue, the current location of your cursor is red. Navigating and searching are easy. Worth the install.

Access to specific man page section
You can view the man pages from section five by passing the section number as an argument to the man command

Check if x86 Solaris based system is 32bit or 64bit
This is likely only valid on Solaris based systems. Unfortunately a lot of the more universal techniques for determining if a system is 32bit or 64bit on x86 solaris fail to give much more information than "i86pc"

Save the list of all available commands in your box to a file
When you press TAB twice in your prompt, bash tells you something like "Display all 4567 possibilities? (y or n)" But when you press "y" you only get the list in the terminal output and, if you want to save it to a file, you have to copy it by hand from the vterm screen. With this utility you save the list to a file or pipe it to another command at will You can use the file saved list to grep for a particular pattern, useful if you are searching for a command but you only remember a few letters


Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: