Another straightforward use of website scraping that illustrates yet a different approach is a weather forecast tool. Specify a zip code, and this script goes to the Census Bureau to obtain population and latitude/longitude information. It visits AccuWeather to extract the current weather in that region. The Code #!/bin/sh # weather - Reports the weather forecast, including lat/long, for a zip code. llurl="http://www.census.gov/cgi-bin/gazetteer?city=&state=&zip=" wxurl="http://wwwa.accuweather.com" wxurl="$wxurl/adcbin/public/local_index_print.asp?zipcode=" if [ "$1" = "-a" ] ; then size=999; shift else size=5 fi if [ $# -eq 0 ] ; then echo "Usage: $0 [-a] zipcode" >&2 exit 1 fi if [ $size -eq 5 ] ; then echo "" # Get some information on the zip code from the Census Bureau lynx -source "${llurl}$1" | \ sed -n '/^<li><strong>/,/^Location:/p' | \ sed 's/<[^>]*>//g;s/^ //g' fi # The weather forecast itself at accuweather.com lynx -source "${wxurl}$1" | \ sed -n '/<font class="sevendayten">/,/[^[:digit:]]<\/font>/p' | \ sed 's/<[^>]*>//g;s/^ [ ]*//g' | \ uniq | \ head -$size exit 0 How It Works This script provides yet another riff on the idea of using a shell script as a wrapper, though in this case the optional flag primarily changes the amount of information filtered through the head at the end of the pipe. This script also takes advantage of the natural source code organization of the two sites to slice out the population and latitude/longitude data prefixed with the strings <strong> and Location:, respectively, and then it slices out the forecast information wrapped in a sevendayten font container. Running the Script The standard way to invoke this script is to specify the desired zip code. If census information is available for that region, it'll be displayed, and the most recent weather forecast summary will be shown too. Add the -a flag, however, and it skips the census information and reports a full ten-day forecast. The Results $ weather 66207 Zip Code: 66207 PO Name: Shawnee Mission (KS) Population (1990): 13863 Location: 38.957472 N, 94.645193 W Currently at 10:35 PM CLEAR Winds SW at 4 mph. Temp: 28 / RF 26. UV Index 0. A typical winter evening in Kansas: a warm 28 degrees Fahrenheit. Brrrrr. (vía: "Wicked Cool Shell Scripts", capítulo 7) -- Facundo Andrés Bianco (Vando.) GNUPG ID: 0x89C1B42F XMPP: [email protected] omb: identi.ca/vando SoftwareLibre.org.ar BuenosAiresLibre.org.ar