This is a URL ripper which uses bing to search for a dork i.e. “powered by wordpress” and output a list of thousands of wordpress sites. This example was actually setup to rip ning sites but you can adjust accordingly. Written in ruby for linux…
#!/usr/bin/env ruby
require ‘net/http’
require ‘uri’
class Search
def search(search)
search = search.gsub(‘ ‘, ‘+’)
linksinc = Array.new
paths =[‘/search?q=’ + search,
‘/search?q=’ + search + ‘&filt=all&first=11&FORM=PERE’,
‘/search?q=’ + search + ‘&filt=all&first=21&FORM=PERE’,
‘/search?q=’ + search + ‘&filt=all&first=31&FORM=PERE’,
‘/search?q=’ + search + ‘&filt=all&first=41&FORM=PERE’,
‘/search?q=’ + search + ‘&filt=all&first=51&FORM=PERE’,
‘/search?q=’ + search + ‘&filt=all&first=61&FORM=PERE’,
‘/search?q=’ + search + ‘&filt=all&first=71&FORM=PERE’,
‘/search?q=’ + search + ‘&filt=all&first=81&FORM=PERE’,
‘/search?q=’ + search + ‘&filt=all&first=91&FORM=PERE’]
paths.each { |path|
host = ‘www.bing.com’ # The web server
begin
timeout(600) do
http = Net::HTTP.new(host) # Create a connection
headers, body = http.get(path) # Request the file
sleep(1)
Thread.current[“headers”] = headers
Thread.current[“body”] = body
end
rescue Exception
#print ‘e’
#puts ‘Error: – see binglinkrip100.rb ‘ + $!
end
headers = Thread.current[“headers”]
body = Thread.current[“body”]
if headers.nil?
next
end
if headers.code == “200” # Check the status code
#print ‘.’
links = URI.extract(body, [‘http’, ‘https’])
links.each { |l|
if l.include? ‘msn.com’
elsif l.include? ‘microsoft.com’
elsif l.include? ‘live.com’
elsif l.include? ‘bingj.com’
elsif l.include? ‘bing.com’
elsif l.include? ‘bing.net’
elsif l.include? ‘ciao.co.uk’
elsif l.include? ‘yahoo.com’
elsif l.include? ‘Shopzilla.co.uk’
elsif l.include? ‘w3.org’
elsif l.include? ‘microsofttranslator.com’
elsif l.include? ‘google.com’
else
if l.include? ‘/’
linksinc << l
end
end
}
else
#puts “#{headers.code} #{headers.message}”
end
}
#puts ”
return linksinc
end
def randomword
searchtext = %w{ and me signed in a aback abacus abacuses abandon abandoned abandoning abdomen abdominal aberdeen abernathy aberration abhor abhorred abhorrent abide abilities ability abject ablaze able ably abnormally abolutely abomination about above abraham abrasive abrupt abruptly absence absent absently absolute absolutely absolve absorb absorbed absorbing absorbs abstract absurd absurdity abundance abundant abuse abused abusing abysmal abyss ac academic academy acceded acceleration accelerations accelerator accent accents accept acceptable acceptance accepted accepting access accident accidental accidentally accolade accommodate accommodations accompanied accompaniment accompany accompanying accomplish accomplished accomplishing accomplishment accomplishments accord according accordingly accosting account accounts accrue accumulate accumulated accumulates accumulation accuracy accurate accurately accusation accusations accused accusers accustomed acetate ache ached aches achieve achievement achievements aching achingly acid acknowledge acknowledged acknowledgment acorn acquaintance acquainted acquiesce acquire acquired acquiring acquit acquittal acrid across act acted acting action actions active activities activity actors actress acts actual actually acutely ad adapt adapted add added adder adding addison addition additional additive address addressed addresses addressing adds adept adeptly adequate adequately adhere adherence adherent adie adjunct adjust adjusted adjustment administer administering administrating administration administrators admirable admiration admired admiring admission admit admits admitted admitting admonished admonishment admonition ado adopted adoration adored adornments adrian adrift adult adults advance advanced advancement advancevalue advancing advantage advantages adventure adventures adversaries adversary advertise advertisement advertisements advertising advice advise advised adviser advising advisor advocacy aeronautics affair affect affected affection affectionate affects affinity affixed afflicted afford afforded afire aflame afoot afraid africa african aftanoon after afterimage afterlife aftermath afternoon afternoons afterthought afterward afterwards agaden again against age aged ageless agent agents ages aggravating aggression aggressive aghast agiel agile aging agitated ago agonizing agony agree agreeable agreeably agreed agreeing agreement agreements ah aha ahead ahhhh ahold ahp ai aid aided aides ailing ailments aim aimed ain air aircraft airfare airline airlines airplane airplanes airport airs aisle aisles ajar akin al alabaster alameda alamos alarm alarmed alarming alarms alaska albany albeit albert albino album albuquerque alcohol alcove alder aldert ale alec alert alessandra alexander alfred algebra ali alibi alighting align alike alive all allaying allclear allconsuming allegiance alleviate alley alleys alleyway alleyways alliances allied allies allotment allotted allow allowed allowing allows allthose alluded alluding allure alluring ally almost alms aloft alone along alongside aloof aloud alpha alphabetical already also alt altadena altar alter alteration altered alternate alternated alternative alternatives alters although altitude altitudes altogether altruistic altur alumina alvarez always am amarillo amateur amateurish amateurishly amaze amazed amazement amazing ambassador amber ambiguous ambition ambitious ambled ambling amends america american americana amiable amid amino amiss amoeba among amongst amount amounts ample amplifier amplifiers amulet amuse amused amusement amuses amusing an anal analog analogous analogs analogy analysis analyze analyzing anarchy anatomy ancestor ancestors anchored ancient and anderith anekdoty anesthetic anew angeles anger angered angle angled angles angling anglo angrily angry anguish angular animal animals animated ankle ankledeep ankles ann annexed annihilate annnnnnnnnnnnything announce announced announcement announcements announcing annoyance annoyed annoying annual another answer answered answering answers ant antagonize antennas anthill anthills anthropologist anti anticipate anticipated anticipating anticipation antimony antique antiquity ants anus anvil anvils anxiety anxious anxiously any anybody anyhow anymore anyone anything anyway anywhere ao ap apart apartment apathy apex aphid aplenty apologetic apologetically apologize apology appalling apparatus apparent apparently appealed appealing appeals appear appearance appeared appearing appears appellation appendicitis appetite applaud applauded applauding applause apple apples applicant applicants application applications applied apply applying appointed appointment appraised appraising appreciate appreciated appreciation appreciatively apprehension apprendi apprentices approach approached approaches approaching appropriate appropriately approval approve approved approximate approximating approximation april apron apronlike aprons arabian arabic arbitrarily arbitrary arc arcane arced arched archer archers arches architect architects arduous are area areas aren argon argue argued arguing argument arguments arid arise aristophanes arithmetic arlene arm armed armies armload armor armored armorer armorers armories armory armpits arms army aroma aromatic arose around arouse arrange arranged arrangement arrangements arranging array arrayed arrest arrested arrests arrival arrive arrived arrives arriving arrogance arrogant arrogantly arrow arrows arrucha arsenal art artery article articles artifacts artifice artificial artificially artillery artisan artisans artist artistic artists artless arts artwork as ascended ascending ascent ascii ascribe ash ashamed ashen ashes ashtrays aside asinine ask askance asked askin asking asks asleep aspect aspects aspen aspirants aspire aspirin aspirins ass assailant assassin assassins assault assay assays assembled assembly assent asserted assertion assess assessed assesses assigned assignment assist assistance assistant assisted associated association assortment assuage assume assumed assuming assumption assumptions assurance assure assured assuring astonished astonishing astonishment astringent astrology astronomer astronomical astronomy asunder asylum at ate athenaeum atlantic atlantica atmosphere atmospheric atomic atomics atoms atone atop attach attached attachment attachments attack attackand attacked attackers attacking attacks attain attempt attempted attempts attend attendants attended attending attention attentive attentively attest attested attire attired attitude attitudes attract attracted attractive attributes au aubrey audience audiences auditorium auditory aug augment augmented augured aunt aura aurora auspicious austere austerity authentic author authorities authority authorized authors auto autocratic automatic automatically automobile automobiles autonomy autumn avail available avalanche avalanched ave avenida avenue average averaged averaging averse avert averting avoid avoided avoiding aw awaited awaits awake awakened awakening awaking awarded awards aware awareness away awe awed awesome awestruck awful awhile awkward awkwardly awoke axe axes axis axle axles aydindril azrith b baade baba babbled babbling babe babies baby babylonian bachelor bachelors bacher back backbone backcountry backdrop backed background backgrounds backhanded backing backs backside backslid backtrack backward backwards backyard bacon bacteria bacteriophages bacterium bad badash badda bade bader badger badgering badly baffled bag baggage bags bah bait baka baked bakelite baker bakeries bakers bakery balance balanced balancing balcony bald balding baldric baldwin bale balk balked ball balled ballet balloon ballpark ballroom ballrooms balls baloney balsam balta bamboo ban band bandage bandaged bandages banded bandleader bands bang banish banished banishing banishment bank banked banker banking banks banners bannock banquet bantam banter banyuma bar barbara barbed barbs bare bared barefoot barely barfly bargain bargaining barge barged barges baring bark barked barking barley barn baronallai barracks barrage barrel barrels barren barricades barrier barring bars bartender base baseball based basement baser bases bashful basic basically basics basin basis bask basket baskets basking bastard bastards batang batch bath bathed bathing bathroom baths bathtub bathwater bats batteria batteries battering battery batting battle battled battlefield battlefields battles battling bausch bawled bay be beach beaches beacon bead beaded beads beam beamed beaming beams bean beans bear beard bearded bearing bearings bears beast beastly beasts beat beaten beating beats beauties beautiful beautifully beauty became because become becomes becoming bed bedchamber bedcover bedding bedraggled bedridden bedrock bedroll bedrolls bedroom beds bedside beeda beefy been bees beetle befalling befitted befitting before beforehand befriend befuddled beg began begetting beggar beggars begged begging begin beginning begins begun behalf behave behaved behaves behaving behavior behaviour beheading beheld behind behooves being beings belatedly belgian belief beliefs believe believed believers believes believing bell bellies belligerent bellow bellowed bellows bellowsmen bells belly belong belonged belongings belongs beloved below belt belts bench benches bend bended bending beneath beneficial benefit benefits benevolence benjamin bent benzine berated berdine berkeley bernadine bernays bernoulli berries berth beryllium beseeched beset beside besides bessel best bested besting bestow bestselling bet beta betcha bethe betray betrayal betrayed betraying bets better betterinstead betterment betting between bevel bewildered bewildering bewilderment bewitching bewitchingly beyond biceps bicycle bid bidda biddas bidden bidding bide bidi biding bids big bigger biggest bikini bill billed billiard billion billow billowed billowing bin bind binders binds bing binomial bins biologist biologists biology birch birches bird birds birth birthday birthright biscuit biscuits bismol bit bitch bitches bite bites biting bits bitsi bitten bitter bitterly bitterness bittersweet bizarre black blackberry blackboard blackboards blackened blackness blacknose blacks blacksmith blacksmiths bladder blade bladed blades blah blame blamed blanch blanched bland blank blanket blanketed blanketing blankets blaspheme blasphemy blast blasted blastomeres blastospheres blasts blaze blazed blazes blazing bleached bleak bleakness bled bleed bleeding blend bless blessed blessing blew blind blinded blindfolded blinding blindingly blindly blindness bling blink blinked blinking bliss blissful blissfully blister blistered blizzard bllllloooooowwwww blobs block blocked blocks blocky blond blonde blood bloodbath bloodcurdling blooded bloodhound bloodhounds bloodied bloodless bloodlines bloodred bloodshed bloodshot bloodstained bloodsucking bloodthirsty bloody bloom blossom blotches blotted blow blowing blown blows blubbering bludgeon bludgeoned bludgeoning blue blueprint blueprints bluff bluffing bluffs bluish blunder blundered blunt blur blurred blurring blurry blurt blurted blush blushed blushing bluuuuurp boar board boarded boards boast boasted boastful boasting boasts boat bob bobbed bobbing bobcat boded bodice bodied bodies bodily body bodyguard boehm bog bohr boil boiled boiling boils boisterous bold bolder boldly boldness bolt bolted bolts bomb bombardments bombs bond bondage bonded bone bones bong bongo bongos bonus bony booda boodas book bookmakers books bookshelf bookstores boom boomed booooo boost boosted boot booth booths boots booty bore bored boredom boring born borne boron borrow borrowed borrowing bosom boss boston both bother bothered bothering bothers botticelli bottle bottled bottles bottom bottomland bottomless bottoms bough boughs bought boulder boulders boulevard boulevards bounced bouncing bound boundaries boundary bounded bounding boundless bounds bouquet bouts bovary bow bowed bowing bowl bowler bowlful box boxes boy boyfriend boyhood boys braces bracing bracken bracket bradley bragged braid braided braids brain brains brake bralda branch branches brandished brandishing brasileiro brass brat bravado brave bravery braving brawl brawls brawny brazil brazilian breached breaching bread break breakfast breakfasts breaking breakneck breaks breakthrough breast breasts breath breathe breathed breathing breaths breathtaking breathy bred breeze breezes breezy brek brewster bribe bribed bribes brick bricklike brickness bricks bride bridge bridged bridges bridle brief briefcase briefly bright brighten brightened brightening brighteyed brightly brightness briiing brilliant brim brimmed brimming brine bring bringer bringing brings brink bristled bristling british brittle broach broached broad broadcast broadened broadway brocade broiling broke broken bronk bronze brooded brooding brook brookhaven brooklyn broom brothels brother brotherhood brothers brought brow brown browns brows brrra brrrrrrr brrrrrrrrr brrrrrrrup bruise bruised brush brushed brushing brutal brutality brute brutes bruuuuup bshshs bubbled bubbles bubbling bubonic buck bucked bucket buckets bucking buckle buckled bucks buckwheat buddy budget buffalo bug bugged bugs buh build builder builders building buildings built bulb bulbs bulge bulged bulges bulk bulky bull bulletin bullheaded bullies bullion bullnecked bullock bully bulwark bulwarking bum bumblebee bump bumped bumping bumps bunch bunched bundle bundled bundles bunk bunks bunsen buoyant bup burble burbled burbling burden burdens burd
searchphrase = searchtext[rand(searchtext.length)]
end
end
class Array
def shuffle!
size.downto(1) { |n| push delete_at(rand(n)) }
self
end
end
######################################################################
####################### Start Main Program ###########################
######################################################################
$hosts = []
5000.times {
s = Search.new
randomword = s.randomword
results = s.search(‘”a micro-blogging service based” ‘ + randomword)
results.each { |r|
begin
split1 = URI.split(r)
next if split1[2].nil?
ohost = split1[2]
opath = split1[5]
next if ohost.nil?
unless $hosts.include? ohost
if opath.nil?
opath = ‘/’
http = Net::HTTP.new(ohost) # Create a connection
headers, body = http.get(opath) # Request the file
if body.include? ‘<a href=”http://en.wikipedia.org/wiki/Micro-blogging”>’
puts ‘http://’ + ohost + opath + ‘,’ + $hosts.length.to_s
system(‘echo “http://’ + ohost + opath + ‘” >> sites.txt’)
$hosts << ohost
end
else
http = Net::HTTP.new(ohost) # Create a connection
headers, body = http.get(opath) # Request the file
if body.include? ‘<a href=”http://en.wikipedia.org/wiki/Micro-blogging”>’
puts ‘http://’ + ohost + opath + ‘,’ + $hosts.length.to_s
system(‘echo “http://’ + ohost + opath + ‘” >> sites.txt’)
$hosts << ohost
end
end
end
rescue Exception # rescue Exception rescues everything
puts ‘Failed: ‘ + $!
end
}
}
