REBOL [ Title: "Auction Site Scanner" Date: 1-Oct-1999 Version: 1.0.0 File: %auction-scan.r Author: "Sterling Newton and Andrew Grossman" Needs: 2.1.3 Purpose: {Harvest search results from any number of auction sites for a given search. } Email: sterling@rebol.com Category: [web util] Notes: {requires %http-post.r and %parse-form.r from the REBOL.org 'web section} ] do %http-post.r do %parse-form.r iseek: func [ search-text [string!] /local sites page frm request results search output ][ results: make block! [] search: copy search-text sites: [ ;site parse rule http://www.ebay.com/ [some [skip to http://cgi.ebay.com/aw-cgi/eBayISAPI.dll?ViewItem&item= copy url to {">} 2 skip copy text to {} (append results reduce [to-url url text]) | skip to end]] ;["category0" 160] options should be put in as hidden values (not done yet) http://auctions.yahoo.com/show/searchoptions [some [skip to http://auctions.yahoo.com/auction/ copy url to {">} 2 skip copy text to {} (append results reduce [to-url url text]) | skip to end]] ;["auccat" 23336] ; not working yet http://auctions.amazon.com/exec/varzea/subst/home/home.html/002-0581295-4022045 [some [skip to {Auction Info} skip to end | skip to {/exec/varzea/ts/auction-glance/} copy url to {#image} skip thru url 2 skip copy text to (append results reduce [join http://auctions.amazon.com url text]) | skip to {/exec/varzea/ts/auction-glance/} copy url to {">} 2 skip copy text to (append results reduce [join http://auctions.amazon.com url text]) | skip to end]] ] replace/all append search "&" " " "+" foreach [site rule] sites [ page: read site frm: parse-form copy/part start: find page " if not find/match probe frm/action http:// [ either #"/" = first frm/action [ insert frm/action copy/part site find find/tail site "//" "/" ] [ insert frm/action site ] ] either (length? frm/inputs) = 2 [ change next frm/inputs search ] [ print {There are too many inputs. Maybe one of the auction sites changed their search form? Contact the author.} ] print [{Searching } site] either frm/method = 'get [ append frm/action "?" request: replace/all join net-utils/export make object! frm/inputs net-utils/export make object! frm/hidden reduce "^/" "&" replace/all request ": " "=" result: read to-url join frm/action request ] [ result: http-post-form frm/action frm/inputs ] parse result rule ] output: make string! "" foreach [url text] results [ append output rejoin [{
  • } text {
    ^/}] ] print [((length? results) / 2) {results returned.}] replace/all search-text "" write to-file ask "Output file: " rejoin [{REBOLAuction:} search-text {^/REBOLAuction: } search-text {^/

      ^/}output {^/

    ^/^/}] ]