2012 User-Agent Blacklist

Below is my reworked version of Perishable Press’ 2010 User-Agent Blacklist. I’ve also added many more bad user agents to the list. It’s compressed, optimized, syntax corrected, and alphabetized. You should notice some speed improvement over the original.

You can check out the original Perishable Press’ 2010 User-Agent Blacklist here:
http://perishablepress.com/press/2010/08/09/2010-user-agent-blacklist/
I’d like to thank them for their work, for without it, I’d probably would have not created my version.

Here’s my compressed, optimized, syntax corrected, and alphabetized version:

# Deny domain access to spammers and other scumbags
RewriteCond %{HTTP_HOST} !^(127\.0\.0\.0|localhost) [NC]
RewriteCond %{HTTP_USER_AGENT} (\$x0e|\%0[ad]|@\$x|_irc|_works|\+select\+|\+union\+|<\?|1,1,1,|\!susie|0d\ 0a|0wn|3gse|4all|4anything|5\.1;\ xv6875|59\.64\.153\.|72\.171\.0\.138|85\.17\.|88\.0\.106\.|98|a_browser|a1\ site|ab(ac|ach|by|erja|ilon|on?t)|acc(ept|ess|oo)|ac(eftp|me|tive|unetix)|add?ress|ad(opt|SARobot|visor)|agent|ah-ha|aihit|aipbot|aktuelles|al(_viewer|arm|bert|ek|exa\ toolbar;\ \(r1\ 1\.5\)|ltop|ma|ot|pha)|am(erica\ online\ browser\ 1\.1|fi(bi)?|zn_assoc)|an(al|archie|dit|on|search|swer|tivirx)|Ap(acheBench|ollo|pie)|ar(ach|chive|ian|oMATIZED)|aboutoil|as(ps|SORT|ter)|at(ari|HENS|local|om|rax|rop|tach|trib)|autoh|av\ fetch|avsearch|axo[dn]|ba(boom|by|ck|id|li|rry|sichttp|tch)|beau?t|be(come|e|ij|nder)|bi(glotron|lgi|son|tacle|tly)|blaiz|blitz|blo(g(l|scope|zice)|ob|w)|bmclient|boi|bond|bord|boris|bost|bot(\.ara|je|w)|bpimage|br(and|ok|oth|owse(abit|x)|uin)|bsalsa|bsdseek|bu(ilt|lls|mble|nny|sca|si|y)|bwh3|ca(fek|fi|mel|nd|ptu|sper|tch)|ccbot|cd34|ceg|cfnetwork|cgichk|cha(0s|ng|os|r|se\ x)|check(_http|er|only)|ch(ek|erryPicker|ill)|ci(CC|pinet|sco|ta|teseer)|cla(m|ria|w)|Cloak|cl?shttp|clush|cmsworldmap|co(ast|de\.com|gent|ldfusion|ll|m(b|mentreader|mon|pan|patible-)|n(c|duc|tact|trol|type|v)|ol|p[iy]|r(al|e-project|n)|s(mos|ta)|wbot)|cr4nk|cra(ft|lwer|nk|p|wler0|zy)|cres|cs-cz|cuill|CUR(I|l|ry)|custo|cute|cyber|cz(3|x)|da(ily|lvik|S|obot|rk|rwin|ta|ten)|dcbot|dcs|dds\ explorer|de(ep|mon|ps|tect|web|x)|dia(gem|m|vol)|Di(gger|gimarc|ibot|llo|ng|s[cp]|tto)|dlc|do(co|tbot|wnloader)|dr(ag|ec|one)|ds(dl|ok)|DSurf15a|dts|duck|dumb|ea(g|rn|rthcom|sydl)|ebin|echo|edco|efp@gmx|egoto|elnsb5|em(ail|er|pas)|en(cyclo|fi|han|terprise_search|volk)|erck|erocr|ev(entax|ere|il)|ewh|ex(ac|ploit|pre|tra)|eyen|fa(ng|st|vOrg|xo)|fdse|feed(24|hub)|fetch|fi(lan|le(boo|Hound)|map|nd|rs)|fire(bat|download/1\.2pre\ firefox/3\.6|fox/(0|2))|fl(am|ash|exum|ickBot|icky|ip|uffy|y)|fo(cus|oky|rum|rv|st|to|un|xy/1;)|fr(ee|iend|ontpage)|fu(ck|er|tile)|fyber|gais|gal(axy)?bot|gbpl|gecko/200(1|2|6|9042316)|gen(er|i)|geo|get(h|left|r|Smart|w)|ggl|gigabaz|gira|gluc|gnome|go(\!?zilla|forit|ldfire|nzo|rnKer|search)|googl[^.]e|goog[^.]le|goo[^.]gle|go[^.]ogle|g[^.]oogle|google(\ wireless|-image)|got(-it|it)|gra[bf]|greg|gru[bp]|gsa-cra|gsearch|gt::www|guidebot|guruji|gyps|h4x|h4x0r|ha(ha|ilo|rv|sh|tena|x)|he(ad|lm|rit)|hgre|hhjhj@yahoo|hippo|hloader|hmse|ho(lm|ly|mePageSearch|sTalking|tbar\ 4\.4\.5\.0)|hpprint|htt(rack|pclient|pconnect|pdown|plib)|human|huron|hybrid|hyper|iaskspi|ibm\ evv|iccra|ichiro|ICS|ida|ie(/5\.0|auto|mpt|xplore\.exe)|il(ium|se|trov)|in(cyWincy|dy|eturl|fonav|kman|nerpr|spect|suran|tellig|terget)|internet(_explorer|\x|Linkagent|Seer)|intraf|ip(2|sel|tcbot)|Iri(a|lbot|vine)|is(c_sys|ilo|rccrawler|spi)|ja(dy|ka|m|va)|JBH|jenn|jet|jiro|jobo|joc|J-SRD|jupit|just|jyx|kash|kazo|kenjin|kernel|keywo|kfsw|kkma|kmc|know|kosmix|krae|krug|ksibot|ktxn|kum|KWebGet|la(bs|chesis|nshan|po|rbin)|le(ech|ts|xi|xxe)|lftp|lib(by|crawl|web|www)|light|likse|lin(c|gue|kcheck)|li[ns]t|litefeeds|live(door|journal|up)|lmq|lo(ader|cu|ndon|ne|op|rk)|lth_|LWP|lynx|ma(c_f|g-?Net|g[ip]|il\.ru|in|jest|m|na|rketwire|s[cs]|ta)|mvi|mcbot|me(cha|diapartners|mo|talogger|tauri|te)|mi(b/2\.2|crosoft(\.url|_internet_explorer)|do|ggi|ix|ndman|ner|ps|r[ae]|rror|s[st]|zz)|mj12|mlbot|mlm|mo(bilerunner|ge|je|oz|re|rfeus|sill|use)) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (mozilla/(0|1|2|3|4\.61\ \[en\]|firefox|mpf|msie\ (1|2|3|4|5|6\.0-|6\.0b|6\.0;\ Windows\ NT;\ DigExt)|7\.0a1;|7\.0b;|6xpv1|crawler)|ms(nbot-(media|products)|nptc|rbot)|mu(ieblackcat|ltithreaddb|sc)|mvac|mwm|my(_age|app|dog|eng|ie2|search|url)|na(g|me|tionaldirectory|ve?r)|near|net(ants|cach|Carta|craft|crawl|front|info|mech|prospector|Seer|sp|x|z)|neu(ral|t)|new(sbreak|sgatorinbox|srob|t)|next|ng-s|ng/2|ni(ce|kto|mb|nja|nte)|no(g|ko|mad|rb|te)|npbot|nu(se|tch|tex)|nwsp|obje|ocel|octo|odi3|oegp|off(by|line)|om(ea|g|http)|onfo|onyx|OpaL|open(f|ssl|TextSiteCrawler|u)|opera\ (2|3|4|5|6|7)|or(ac|angeBot|bit|eg)|osis|our|outf|owl|p3p_|Pa(ckRat|ge(2rss|fet)|nsci|pa|rser|tw|vu)|pb2pb|pcbrow|pe([ae]r|pe|rfect|rl|rsonaPilot|tit)|phoenix/0\.|php|phras|pi(calo|ff|g|ng|pe|rs|X)|pla(g|ne?t|tform|ystation)|pl(esk|uck|ukkie)|Po(ckey|e-com|irot|mp|st|werset)|pre(load|ss)|privoxy|pro(be|gram_shareware|tect|tocol|wl|xie|xy)|psbot|pt-BR;\ rv:1\.9\.0\.(3|18)\ Firefox/3\.0|pu(bsub|f|lse|nit|rebot|rity)|py(q|th)|que(ry|st)|qweer|ra(dian|mbler|mp|pid|w(dog|grunt))|re(ap|eder|fresh|get|levare|po|qu|se|trieve|volt)|rix|rma|roboz|rocket|rogue|rpt-http|rsscache|ruby|ruff|rufus|rv:0\.9\.7|sa(lt|mple|uger|vvy)|sb(cyds|ide|log|p)|sc(agent|an|ej_)|sch(ed|izo|long|mo)|scorp|sco[tu]t|scrawl|screenshot|se(amonkey/1\.5a|arch(17|bot|me)|eker|ga|mto|nsis|op|pt|tup\.php|zn)|Sha(i|re|rp|z)|sh(el[lo]|erl|im|opwiki)|si(lurian|mple|ph)|site(check|kiosk|scan|vigil|x)|skam|skimp|skygrid|sl(edink|eip|euth|ide|y)|sm(ag|artDownload|urf)|snake|snap(bot|shot)|sni[fp]|snoop|soc(k|sci)|so(gou|hu|lr|me|so)|sp(a[dn]|bot|eed|egla|here|icon|[iy]der|in|roose|url|utnik)|sq(lmap|ui|wid|worm)|ssm_ag|st(ack|amp|ate|eel|ilo|rateg|ress|yle)|su(bot|c[hk]|me|nos\ 5\.7|nrise|per(bot|bro|HTTP|vi)|r(f4me|fbot|vey)|si|z[au])|sweep|sy(gol|hunt|napse|nc2it|stems)|szukacz|ta(g(ger|oo|yu)|ke|lkro|mu|ndem|rantula)|tbot|tcf|tcs/1|te(amsoft|comi|esoft|le(port|soft)|mpleton|ncent|rrawiz|st|xnut)|thomas|ti(ehttp|me(bot|ly)|pp|scali|tan)|tmcrawler|tmhtload|to(crawl|dobr|ngco|olbar;\ r1|olpak|pic|pyx|rrent)|tr(ack|anslate|aveler|eeview|icus|ue)|tu(nnel|ring|rnitin|torgig)|TV33_Mercator|tw(at|eak|ice)|tygo|uchoo|UIowaCrawler|ultraseek|un(avail|f|iversal)|upg1|uptime|url(base|lib|y)|user-?agent:?|usyd|UtilMind|va(cuum|gabo|let|mp|yala)|vci|ver(i~li|if|sus)|vi(a|rtual|sibilitygap|sual)|void|voyager|vsyn|w000?0?t|w3(af|mir|search)|wa(lhello|lk|nd|ol|tch|vefire)|wbdbot|weather|web(\.ima|\.by\.mail|2mal|Auto|bot|cat|collage|cor|crawl|dat|dup|go|Hook|is|itpr|lea|min|mole|money|p|ql|robot|ster|surf|tre|vac|Washer|weasel|zip)|wells|wep_s|wget|Wha(cker|tWeb)|whiz|wi(dow|SEbot|sh|zz)|win(67|dows-rss|dows\ 3|dows\ 95|dows\ me|dows\ NT\ 6\.1;\ tr;\ rv:1\.9\.2\.6|ht|odws)|Wonder|wor(dp|io|ks|ld|th)|Wweb|WWW(-Mechanize|c|o|ster)|WUMPUS|x-Tractor|xaldon|xbot|xenu|XGET|xirq|xpymep|yacy|yahoo(-mmaudvid|yseeker|ysmcm)|yamm|yan[dg]|yo(ono|ri|tta)|yplus\ |ytunnel|zade|zagre|ze(al|bot|rx|us|W)|zhuaxia|zipcode|zixy|zmao|ZmEu) [NC]
RewriteRule .* - [G]

2-2-12 Updated to allow Firefox 10.0 through.
3-12-12 Updated. Removed ‘rip’, false positive with iPhone.

3-12-12 Added a more reorganized version by grouping.
If you use this version and you’re on Apach 2.2 or higher, I recommend using non-capturing groups. Done using ‘?:’

# Deny domain access to spammers and other scumbags
RewriteCond %{HTTP_HOST} !^(127\.0\.0\.0|localhost) [NC]
RewriteCond %{HTTP_USER_AGENT} (\$x0e|\%0[ad]|@\$x|_irc|_works|\+select\+|\+union\+|<\?|1,1,1,|\!susie|0d\ 0a|0wn|3gse|4all|4anything|5\.1;\ xv6875|59\.64\.153\.|72\.171\.0\.138|85\.17\.|88\.0\.106\.|98) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (a_browser|a1\ site|ab(ac|ach|by|erja|ilon|on?t)|acc(ept|ess|oo)|ac(eftp|me|tive|unetix)|add?ress|ad(opt|SARobot|visor)|agent|ah-ha|aihit|aipbot|aktuelles|al(_viewer|arm|bert|ek|exa\ toolbar;\ \(r1\ 1\.5\)|ltop|ma|ot|pha)|am(erica\ online\ browser\ 1\.1|fi(bi)?|zn_assoc)|an(al|archie|dit|on|search|swer|tivirx)|Ap(acheBench|ollo|pie)|ar(ach|chive|ian|oMATIZED)|aboutoil|as(ps|SORT|ter)|at(ari|HENS|local|om|rax|rop|tach|trib)|autoh|av\ fetch|avsearch|axo[dn]) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (ba(boom|by|ck|id|li|rry|sichttp|tch)|beau?t|be(come|e|ij|nder)|bi(glotron|lgi|son|tacle|tly)|blaiz|blitz|blo(g(l|scope|zice)|ob|w)|bmclient|boi|bond|bord|boris|bost|bot(\.ara|je|w)|bpimage|br(and|ok|oth|owse(abit|x)|uin)|bsalsa|bsdseek|bu(ilt|lls|mble|nny|sca|y)|bwh3) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (ca(fek|fi|mel|nd|ptu|sper|tch)|ccbot|cd34|ceg|cfnetwork|cgichk|cha(0s|ng|os|r|se\ x)|check(_http|er|only)|ch(ek|erryPicker|ill)|ci(CC|pinet|sco|ta|teseer)|cla(m|ria|w)|Cloak|cl?shttp|clush|cmsworldmap|co(ast|de\.com|gent|ldfusion|ll|m(b|mentreader|mon|pan|patible-)|n(c|duc|tact|trol|type|v)|ol|p[iy]|r(al|e-project|n)|s(mos|ta)|wbot)|cr4nk|cra(ft|lwer|nk|p|wler0|zy)|cres|cs-cz|cuill|CUR(I|l|ry)|custo|cute|cyber|cz(3|x)) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (da(ily|lvik|S|obot|rk|rwin|ta|ten)|dcbot|dcs|dds\ explorer|de(ep|mon|ps|tect|web|x)|dia(gem|m|vol)|Di(gger|gimarc|ibot|llo|ng|s[cp]|tto)|dlc|do(co|tbot|wnloader)|dr(ag|ec|one)|ds(dl|ok)|DSurf15a|dts|duck|dumb) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (ea(g|rn|rthcom|sydl)|ebin|echo|edco|efp@gmx|egoto|elnsb5|em(ail|er|pas)|en(cyclo|fi|han|terprise_search|volk)|erck|erocr|ev(entax|ere|il)|ewh|ex(ac|ploit|pre|tra)|eyen) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (fa(ng|st|vOrg|xo)|fdse|feed(24|hub)|fetch|fi(lan|le(boo|Hound)|map|nd|rs)|fire(bat|download/1\.2pre\ firefox/3\.6|fox/(0|1\.|2))|fl(am|ash|exum|ickBot|icky|ip|uffy|y)|fo(cus|oky|rum|rv|st|to|un|xy/1;)|fr(ee|iend|ontpage)|fu(ck|er|tile)|fyber) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (gais|gal(axy)?bot|gbpl|gecko/200(1|2|6|9042316)|gen(er|i)|geo|get(h|left|r|Smart|w)|ggl|gigabaz|gira|gluc|gnome|go(\!?zilla|forit|ldfire|nzo|rnKer|search)|googl[^.]e|goog[^.]le|goo[^.]gle|go[^.]ogle|g[^.]oogle|google(\ wireless|-image)|got(-it|it)|gra[bf]|greg|gru[bp]|gsa-cra|gsearch|gt::www|guidebot|guruji|gyps) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (h4x|h4x0r|ha(ha|ilo|rv|sh|tena|x)|he(ad|lm|rit)|hgre|hhjhj@yahoo|hippo|hloader|hmse|ho(lm|ly|mePageSearch|sTalking|tbar\ 4\.4\.5\.0)|hpprint|htt(rack|pclient|pconnect|pdown|plib)|human|huron|hybrid|hyper) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (iaskspi|ibm\ evv|iccra|ichiro|ICS|ida|ie(/5\.0|auto|mpt|xplore\.exe)|il(ium|se|trov)|in(cyWincy|dy|eturl|fonav|kman|nerpr|spect|suran|tellig|terget)|internet(_explorer|\x|Linkagent|Seer)|intraf|ip(2|sel|tcbot)|Iri(a|lbot|vine)|is(c_sys|ilo|rccrawler|spi) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (ja(dy|ka|m|va)|JBH|jenn|jet|jiro|jobo|jo(c|rgee)|J-SRD|jupit|just|jyx|kash|kazo|kenjin|kernel|keywo|kfsw|kkma|kmc|know|kosmix|krae|krug|ksibot|ktxn|kum|KWebGet|la(bs|chesis|nshan|po|rbin)|le(ech|ts|xi|xxe)|lftp|lib(by|crawl|web|www)|light|likse|lin(c|gue|kcheck)|li[ns]t|litefeeds|live(door|journal|up)|lmq|lo(ader|cu|ndon|ne|op|rk)|lth_|LWP|lynx) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (ma(c_f|g-?Net|g[ip]|il\.ru|in|jest|m|na|rketwire|s[cs]|ta)|mvi|mcbot|me(cha|diapartners|mo|talogger|tauri|te)|mi(b/2\.2|crosoft(\.url|_internet_explorer)|do|ggi|ix|ndman|ner|ps|r[ae]|rror|s[st]|zz)|mj12|mlbot|mlm|mo(bilerunner|ge|je|oz|re|rfeus|sill|use)|mozilla/(0|1|2|3|4\.61\ \[en\]|firefox|mpf|msie\ (1|2|3|4|5|6\.0-|6\.0b|6\.0;\ Windows\ NT;\ DigExt)|7\.0a1;|7\.0b;|6xpv1|crawler)|mra|ms(nbot-(media|products)|nptc|rbot)|mu(ieblackcat|ltithreaddb|sc)|mvac|mwm|my(_age|app|dog|eng|ie2|search|url) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (na(g|me|tionaldirectory|ve?r)|near|net(ants|cach|Carta|craft|crawl|front|info|mech|prospector|Seer|sp|x|z)|neu(ral|t)|new(sbreak|sgatorinbox|srob|t)|next|ng-s|ng/2|ni(ce|kto|mb|nja|nte)|no(g|ko|mad|rb|te)|npbot|nu(se|tch|tex)|nwsp|obje|ocel|octo|odi3|oegp|off(by|line)|om(ea|g|http)|onfo|onyx|OpaL|open(f|ssl|TextSiteCrawler|u)|opera\ (2|3|4|5|6|7)|or(ac|angeBot|bit|eg)|osis|our|outf|owl) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (p3p_|Pa(ckRat|ge(2rss|fet)|nsci|pa|rser|tw|vu)|pb2pb|pcbrow|pe([ae]r|pe|rfect|rl|rsonaPilot|tit)|phoenix/0\.|php|phras|pi(calo|ff|g|ng|pe|rs|X)|pla(g|ne?t|tform|ystation)|pl(esk|uck|ukkie)|Po(ckey|e-com|irot|mp|st|werset)|pre(load|ss)|privoxy|pro(be|gram_shareware|tect|tocol|wl|xie|xy)|psbot|pt-BR;\ rv:1\.9\.0\.(3|18)\ Firefox/3\.0|pu(bsub|f|lse|nit|rebot|rity)|py(q|th)|que(ry|st)|qweer|ra(dian|mbler|mp|pid|w(dog|grunt))|re(ap|eder|fresh|get|levare|po|qu|se|trieve|volt)|rix|rma|roboz|rocket|rogue|rpt-http|rsscache|ruby|ruff|rufus|rv:0\.9\.7) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (sa(lt|mple|uger|vvy)|sb(cyds|ide|log|p)|sc(agent|an|ej_)|sch(ed|izo|long|mo)|scorp|sco[tu]t|scrawl|screenshot|se(amonkey/1\.5a|arch(17|bot|me)|eker|ga|mrush|mto|nsis|op|pt|tup\.php|zn)|Sha(i|re|rp|z)|sh(el[lo]|erl|im|opwiki)|si(lurian|mple|ph)|site(check|kiosk|scan|vigil|x)|skam|skimp|skygrid|sl(edink|eip|euth|ide|y)|sm(ag|artDownload|urf)|snake|snap(bot|shot)|sni[fp]|snoop|soc(k|sci)|so(gou|hu|lr|me|so)|sp(a[dn]|bot|eed|egla|here|icon|[iy]der|in|roose|url|utnik)|sq(lmap|ui|wid|worm)|ssm_ag|st(ack|amp|ate|eel|ilo|rateg|ress|yle)|su(bot|c[hk]|me|nos\ 5\.7|nrise|per(bot|bro|HTTP|vi)|r(f4me|fbot|vey)|si|z[au])|swe(bot|ep)|sy(gol|hunt|napse|nc2it|stems)|szukacz) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (ta(g(ger|oo|yu)|ke|lkro|mu|ndem|rantula)|tbot|tcf|tcs/1|te(amsoft|comi|esoft|le(port|soft)|mpleton|ncent|rrawiz|st|xnut)|thomas|ti(ehttp|me(bot|ly)|pp|scali|tan)|tmcrawler|tmhtload|to(crawl|dobr|ngco|olbar;\ r1|olpak|pic|pyx|rrent)|tr(ack|anslate|aveler|eeview|icus|ue)|tu(nnel|ring|rnitin|torgig)|TV33_Mercator|tw(at|eak|ice)|tygo) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (uchoo|UIowaCrawler|ultraseek|un(avail|f|iversal)|upg1|uptime|url(base|lib|y)|user-?agent:?|usyd|UtilMind|va(cuum|gabo|let|mp|yala)|vci|ver(i~li|if|sus)|vi(a|rtual|sibilitygap|sual)|void|voyager|vsyn) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (w000?0?t|w3(af|mir|search)|wa(lhello|lk|nd|ol|tch|vefire)|wbdbot|weather|web(\.ima|\.by\.mail|2mal|Auto|bot|cat|collage|cor|crawl|dat|dup|go|Hook|is|itpr|lea|min|mole|money|p|ql|robot|ster|surf|tre|vac|Washer|weasel|zip)|wells|wep_s|wget|Wha(cker|tWeb)|whiz|wi(dow|SEbot|sh|zz)|win(67|dows-rss|dows\ (3|9[58]|me|NT\ 6\.1;\ tr;\ rv:1\.9\.2\.6)|ht|odws)|Wonder|wor(dp|io|ks|ld|th)|Wweb|WWW(-Mechanize|c|o|ster)|WUMPUS) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (x-Tractor|xaldon|xbot|xenu|XGET|xirq|xpymep|yacy|yahoo(-mmaudvid|yseeker|ysmcm)|yamm|yan[dg]|yo(ono|ri|tta)|yplus\ |ytunnel|zade|zagre|ze(al|bot|rx|us|W)|zhuaxia|zipcode|zixy|zmao|ZmEu) [NC]

RewriteRule .* - [G]

Reorganized version with non-capturing groups (use of ?:) Apache 2.x and up only!

# Deny domain access to spammers and other scumbags
RewriteCond %{HTTP_HOST} !^(?:127\.0\.0\.0|localhost) [NC]
RewriteCond %{HTTP_USER_AGENT} (?:\$x0e|\%0[ad]|@\$x|_irc|_works|\+select\+|\+union\+|<\?|1,1,1,|\!susie|0d\ 0a|0wn|3gse|4all|4anything|5\.1;\ xv6875|59\.64\.153\.|72\.171\.0\.138|85\.17\.|88\.0\.106\.|98) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (?:a_browser|a1\ site|ab(?:ac|ach|by|erja|ilon|on?t)|acc(?:ept|ess|oo)|ac(?:eftp|me|tive|unetix)|add?ress|ad(?:opt|SARobot|visor)|agent|ah-ha|aihit|aipbot|aktuelles|al(?:_viewer|arm|bert|ek|exa\ toolbar;\ \(?:r1\ 1\.5\)|ltop|ma|ot|pha)|am(?:erica\ online\ browser\ 1\.1|fi(?:bi)?|zn_assoc)|an(?:al|archie|dit|on|search|swer|tivirx)|Ap(?:acheBench|ollo|pie)|ar(?:ach|chive|ian|oMATIZED)|aboutoil|as(?:ps|SORT|ter)|at(?:ari|HENS|local|om|rax|rop|tach|trib)|autoh|av\ fetch|avsearch|axo[dn]) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (?:ba(?:boom|by|ck|id|li|rry|sichttp|tch)|beau?t|be(?:come|e|ij|nder)|bi(?:glotron|lgi|son|tacle|tly)|blaiz|blitz|blo(?:g(?:l|scope|zice)|ob|w)|bmclient|boi|bond|bord|boris|bost|bot(?:\.ara|je|w)|bpimage|br(?:and|ok|oth|owse(?:abit|x)|uin)|bsalsa|bsdseek|bu(?:ilt|lls|mble|nny|sca|y)|bwh3) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (?:ca(?:fek|fi|mel|nd|ptu|sper|tch)|ccbot|cd34|ceg|cfnetwork|cgichk|cha(?:0s|ng|os|r|se\ x)|check(?:_http|er|only)|ch(?:ek|erryPicker|ill)|ci(?:CC|pinet|sco|ta|teseer)|cla(?:m|ria|w)|Cloak|cl?shttp|clush|cmsworldmap|co(?:ast|de\.com|gent|ldfusion|ll|m(?:b|mentreader|mon|pan|patible-)|n(?:c|duc|tact|trol|type|v)|ol|p[iy]|r(?:al|e-project|n)|s(?:mos|ta)|wbot)|cr4nk|cra(?:ft|lwer|nk|p|wler0|zy)|cres|cs-cz|cuill|CUR(?:I|l|ry)|custo|cute|cyber|cz(?:3|x)) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (?:da(?:ily|lvik|S|obot|rk|rwin|ta|ten)|dcbot|dcs|dds\ explorer|de(?:ep|mon|ps|tect|web|x)|dia(?:gem|m|vol)|Di(?:gger|gimarc|ibot|llo|ng|s[cp]|tto)|dlc|do(?:co|tbot|wnloader)|dr(?:ag|ec|one)|ds(?:dl|ok)|DSurf15a|dts|duck|dumb) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (?:ea(?:g|rn|rthcom|sydl)|ebin|echo|edco|efp@gmx|egoto|elnsb5|em(?:ail|er|pas)|en(?:cyclo|fi|han|terprise_search|volk)|erck|erocr|ev(?:entax|ere|il)|ewh|ex(?:ac|ploit|pre|tra)|eyen) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (?:fa(?:ng|st|vOrg|xo)|fdse|feed(?:24|hub)|fetch|fi(?:lan|le(?:boo|Hound)|map|nd|rs)|fire(?:bat|download/1\.2pre\ firefox/3\.6|fox/(?:0|1\.|2))|fl(?:am|ash|exum|ickBot|icky|ip|uffy|y)|fo(?:cus|oky|rum|rv|st|to|un|xy/1;)|fr(?:ee|iend|ontpage)|fu(?:ck|er|tile)|fyber) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (?:gais|gal(?:axy)?bot|gbpl|gecko/200(?:1|2|6|9042316)|gen(?:er|i)|geo|get(?:h|left|r|Smart|w)|ggl|gigabaz|gira|gluc|gnome|go(?:\!?zilla|forit|ldfire|nzo|rnKer|search)|googl[^.]e|goog[^.]le|goo[^.]gle|go[^.]ogle|g[^.]oogle|google(?:\ wireless|-image)|got(-?:it|it)|gra[bf]|greg|gru[bp]|gsa-cra|gsearch|gt::www|guidebot|guruji|gyps) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (?:h4x|h4x0r|ha(?:ha|ilo|rv|sh|tena|x)|he(?:ad|lm|rit)|hgre|hhjhj@yahoo|hippo|hloader|hmse|ho(?:lm|ly|mePageSearch|sTalking|tbar\ 4\.4\.5\.0)|hpprint|htt(?:rack|pclient|pconnect|pdown|plib)|human|huron|hybrid|hyper) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (?:iaskspi|ibm\ evv|iccra|ichiro|ICS|ida|ie(?:/5\.0|auto|mpt|xplore\.exe)|il(?:ium|se|trov)|in(?:cyWincy|dy|eturl|fonav|kman|nerpr|spect|suran|tellig|terget)|internet(?:_explorer|\x|Linkagent|Seer)|intraf|ip(?:2|sel|tcbot)|Iri(?:a|lbot|vine)|is(?:c_sys|ilo|rccrawler|spi) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (?:ja(?:dy|ka|m|va)|JBH|jenn|jet|jiro|jobo|jo(?:c|rgee)|J-SRD|jupit|just|jyx|kash|kazo|kenjin|kernel|keywo|kfsw|kkma|kmc|know|kosmix|krae|krug|ksibot|ktxn|kum|KWebGet|la(?:bs|chesis|nshan|po|rbin)|le(?:ech|ts|xi|xxe)|lftp|lib(?:by|crawl|web|www)|light|likse|lin(?:c|gue|kcheck)|li[ns]t|litefeeds|live(?:door|journal|up)|lmq|lo(?:ader|cu|ndon|ne|op|rk)|lth_|LWP|lynx) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (?:ma(?:c_f|g-?Net|g[ip]|il\.ru|in|jest|m|na|rketwire|s[cs]|ta)|mvi|mcbot|me(?:cha|diapartners|mo|talogger|tauri|te)|mi(?:b/2\.2|crosoft(?:\.url|_internet_explorer)|do|ggi|ix|ndman|ner|ps|r[ae]|rror|s[st]|zz)|mj12|mlbot|mlm|mo(?:bilerunner|ge|je|oz|re|rfeus|sill|use)|mozilla/(?:0|1|2|3|4\.61\ \[en\]|firefox|mpf|msie\ (?:1|2|3|4|5|6\.0-|6\.0b|6\.0;\ Windows\ NT;\ DigExt)|7\.0a1;|7\.0b;|6xpv1|crawler)|mra|ms(?:nbot-(?:media|products)|nptc|rbot)|mu(?:ieblackcat|ltithreaddb|sc)|mvac|mwm|my(?:_age|app|dog|eng|ie2|search|url) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (?:na(?:g|me|tionaldirectory|ve?r)|near|net(?:ants|cach|Carta|craft|crawl|front|info|mech|prospector|Seer|sp|x|z)|neu(?:ral|t)|new(?:sbreak|sgatorinbox|srob|t)|next|ng-s|ng/2|ni(?:ce|kto|mb|nja|nte)|no(?:g|ko|mad|rb|te)|npbot|nu(?:se|tch|tex)|nwsp|obje|ocel|octo|odi3|oegp|off(?:by|line)|om(?:ea|g|http)|onfo|onyx|OpaL|open(?:f|ssl|TextSiteCrawler|u)|opera\ (?:2|3|4|5|6|7)|or(?:ac|angeBot|bit|eg)|osis|our|outf|owl) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (?:p3p_|Pa(?:ckRat|ge(?:2rss|fet)|nsci|pa|rser|tw|vu)|pb2pb|pcbrow|pe(?:[ae]r|pe|rfect|rl|rsonaPilot|tit)|phoenix/0\.|php|phras|pi(?:calo|ff|g|ng|pe|rs|X)|pla(?:g|ne?t|tform|ystation)|pl(?:esk|uck|ukkie)|Po(?:ckey|e-com|irot|mp|st|werset)|pre(?:load|ss)|privoxy|pro(?:be|gram_shareware|tect|tocol|wl|xie|xy)|psbot|pt-BR;\ rv:1\.9\.0\.(?:3|18)\ Firefox/3\.0|pu(?:bsub|f|lse|nit|rebot|rity)|py(?:q|th)|que(?:ry|st)|qweer|ra(?:dian|mbler|mp|pid|w(?:dog|grunt))|re(?:ap|eder|fresh|get|levare|po|qu|se|trieve|volt)|rix|rma|roboz|rocket|rogue|rpt-http|rsscache|ruby|ruff|rufus|rv:0\.9\.7) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (?:sa(?:lt|mple|uger|vvy)|sb(?:cyds|ide|log|p)|sc(?:agent|an|ej_)|sch(?:ed|izo|long|mo)|scorp|sco[tu]t|scrawl|screenshot|se(?:amonkey/1\.5a|arch(?:17|bot|me)|eker|ga|mrush|mto|nsis|op|pt|tup\.php|zn)|Sha(?:i|re|rp|z)|sh(?:el[lo]|erl|im|opwiki)|si(?:lurian|mple|ph)|site(?:check|kiosk|scan|vigil|x)|skam|skimp|skygrid|sl(?:edink|eip|euth|ide|y)|sm(?:ag|artDownload|urf)|snake|snap(?:bot|shot)|sni[fp]|snoop|soc(?:k|sci)|so(?:gou|hu|lr|me|so)|sp(?:a[dn]|bot|eed|egla|here|icon|[iy]der|in|roose|url|utnik)|sq(?:lmap|ui|wid|worm)|ssm_ag|st(?:ack|amp|ate|eel|ilo|rateg|ress|yle)|su(?:bot|c[hk]|me|nos\ 5\.7|nrise|per(?:bot|bro|HTTP|vi)|r(?:f4me|fbot|vey)|si|z[au])|swe(?:bot|ep)|sy(?:gol|hunt|napse|nc2it|stems)|szukacz) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (?:ta(?:g(?:ger|oo|yu)|ke|lkro|mu|ndem|rantula)|tbot|tcf|tcs/1|te(?:amsoft|comi|esoft|le(?:port|soft)|mpleton|ncent|rrawiz|st|xnut)|thomas|ti(?:ehttp|me(?:bot|ly)|pp|scali|tan)|tmcrawler|tmhtload|to(?:crawl|dobr|ngco|olbar;\ r1|olpak|pic|pyx|rrent)|tr(?:ack|anslate|aveler|eeview|icus|ue)|tu(?:nnel|ring|rnitin|torgig)|TV33_Mercator|tw(?:at|eak|ice)|tygo) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (?:uchoo|UIowaCrawler|ultraseek|un(?:avail|f|iversal)|upg1|uptime|url(?:base|lib|y)|user-?agent:?|usyd|UtilMind|va(?:cuum|gabo|let|mp|yala)|vci|ver(?:i~li|if|sus)|vi(?:a|rtual|sibilitygap|sual)|void|voyager|vsyn) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (?:w000?0?t|w3(?:af|mir|search)|wa(?:lhello|lk|nd|ol|tch|vefire)|wbdbot|weather|web(?:\.ima|\.by\.mail|2mal|Auto|bot|cat|collage|cor|crawl|dat|dup|go|Hook|is|itpr|lea|min|mole|money|p|ql|robot|ster|surf|tre|vac|Washer|weasel|zip)|wells|wep_s|wget|Wha(?:cker|tWeb)|whiz|wi(?:dow|SEbot|sh|zz)|win(?:67|dows-rss|dows\ (?:3|9[58]|me|NT\ 6\.1;\ tr;\ rv:1\.9\.2\.6)|ht|odws)|Wonder|wor(?:dp|io|ks|ld|th)|Wweb|WWW(?:-Mechanize|c|o|ster)|WUMPUS) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (?:x-Tractor|xaldon|xbot|xenu|XGET|xirq|xpymep|yacy|yahoo(?:-mmaudvid|yseeker|ysmcm)|yamm|yan[dg]|yo(?:ono|ri|tta)|yplus\ |ytunnel|zade|zagre|ze(?:al|bot|rx|us|W)|zhuaxia|zipcode|zixy|zmao|ZmEu) [NC]

RewriteRule .* - [G]
Advertisements
Published in: on October 16, 2011 at 10:59 am  Leave a Comment  

The URI to TrackBack this entry is: https://tomolivercv.wordpress.com/2011/10/16/2012-user-agent-blacklist/trackback/

%d bloggers like this: