User-agent: ,bot Disallow: / User-agent: .bot Disallow: / User-agent: /bot Disallow: / User-agent: :bot Disallow: / User-agent: ;bot Disallow: / User-agent: +bot Disallow: / User-agent: \bot Disallow: / User-agent: acme.spider Disallow: / User-agent: AgentName/0.1libwww-perl/6.02 Disallow: / User-agent: alexa Disallow: / User-agent: Alexandria prototype project Disallow: / User-agent: Alexandria prototype+project Disallow: / User-agent: Alexandria+prototype project Disallow: / User-agent: Alexandria+prototype+project Disallow: / User-agent: AllenTrack Disallow: / User-agent: almaden Disallow: / User-agent: appie Disallow: / User-agent: Arachmo Disallow: / User-agent: architext Disallow: / User-agent: archive.org_bot Disallow: / User-agent: arks Disallow: / User-agent: asterias Disallow: / User-agent: atomz Disallow: / User-agent: autoemailspider Disallow: / User-agent: awbot Disallow: / User-agent: Baiduspider Disallow: / User-agent: baiduspider Disallow: / User-agent: bbot Disallow: / User-agent: BDFetch Disallow: / User-agent: biadu Disallow: / User-agent: biglotron Disallow: / User-agent: bingbot Crawl-Delay: 3000 User-agent: bjaaland Disallow: / User-agent: blaiz-bee Disallow: / User-agent: bloglines Disallow: / User-agent: blogpulse Disallow: / User-agent: boitho.com-dc Disallow: / User-agent: bookmark-manager Disallow: / User-agent: bot Disallow: / User-agent: -bot Disallow: / User-agent: Brutus/AET Disallow: / User-agent: bspider Disallow: / User-agent: bwh3_user_agent Disallow: / User-agent: celestial Disallow: / User-agent: cfnetwork|checkbot Disallow: / User-agent: checkprivacy Disallow: / User-agent: ChinasLocalsBrowses2.6 Disallow: / User-agent: cloakDetect Disallow: / User-agent: CodesSamplesWebsClient Disallow: / User-agent: combine Disallow: / User-agent: commons-httpclient Disallow: / User-agent: contentmatch Disallow: / User-agent: ContentSmartz Disallow: / User-agent: core Disallow: / User-agent: CoverScout Disallow: / User-agent: crawl Disallow: / User-agent: crawler Disallow: / User-agent: cursor Disallow: / User-agent: custo Disallow: / User-agent: DataCha0s/2.0 Disallow: / User-agent: daumoa Disallow: / User-agent: DemosBot Disallow: / User-agent: docomo Disallow: / User-agent: domain-crawler Disallow: / User-agent: Download+Master Disallow: / User-agent: DSurf Disallow: / User-agent: dtSearchSpider Disallow: / User-agent: dumbot Disallow: / User-agent: easydl Disallow: / User-agent: EmailSiphon Disallow: / User-agent: EmailWolf Disallow: / User-agent: exabot Disallow: / User-agent: fast-webcrawler Disallow: / User-agent: favorg Disallow: / User-agent: FDM 1 Disallow: / User-agent: FDM+1 Disallow: / User-agent: feedburner Disallow: / User-agent: FeedFetcher Disallow: / User-agent: ferret Disallow: / User-agent: Fetch API Request Disallow: / User-agent: Fetch API+Request Disallow: / User-agent: Fetch+API Request Disallow: / User-agent: Fetch+API+Request Disallow: / User-agent: findlinks Disallow: / User-agent: Fulltext Disallow: / User-agent: Funnelback Disallow: / User-agent: gaisbot Disallow: / User-agent: GetRight Disallow: / User-agent: geturl Disallow: / User-agent: gigabot Disallow: / User-agent: girafabot Disallow: / User-agent: gnodspider Disallow: / User-agent: Goldfire Server Disallow: / User-agent: Goldfire+Server Disallow: / User-agent: grub Disallow: / User-agent: gulliver Disallow: / User-agent: harvest Disallow: / User-agent: heritrix Disallow: / User-agent: hl_ftien_spider Disallow: / User-agent: holmes Disallow: / User-agent: htdig Disallow: / User-agent: htmlparser Disallow: / User-agent: HttpComponents/1.1 Disallow: / User-agent: HTTPFetcher Disallow: / User-agent: httpget?5.2.2 Disallow: / User-agent: httpget-5.2.2 Disallow: / User-agent: httrack Disallow: / User-agent: HuaweiSymantecSpider Disallow: / User-agent: ia_archiver Disallow: / User-agent: ICC-Crawler Disallow: / User-agent: ichiro Disallow: / User-agent: iktomi Disallow: / User-agent: ilse Disallow: / User-agent: internetseer Disallow: / User-agent: intute Disallow: / User-agent: iSiloX Disallow: / User-agent: Jakarta+Commons-HttpClient Disallow: / User-agent: java Disallow: / User-agent: jeeves Disallow: / User-agent: jobo Disallow: / User-agent: kyluka Disallow: / User-agent: larbin Disallow: / User-agent: libcurl Disallow: / User-agent: libwww Disallow: / User-agent: libwww-perl Disallow: / User-agent: lilina Disallow: / User-agent: linkbot Disallow: / User-agent: linkcheck Disallow: / User-agent: linkchecker Disallow: / User-agent: LinkLint-checkonly Disallow: / User-agent: linkscan Disallow: / User-agent: linkwalker Disallow: / User-agent: livejournal.com Disallow: / User-agent: lmspider Disallow: / User-agent: LOCKSS Disallow: / User-agent: lwp Disallow: / User-agent: LWP::Simple Disallow: / User-agent: lwp-request Disallow: / User-agent: lwp-tivial Disallow: / User-agent: lwp-trivial Disallow: / User-agent: lycos_ Disallow: / User-agent: lycos+ Disallow: / User-agent: mail.ru Disallow: / User-agent: Mail.RU_Bot/Fast/2.0 Disallow: / User-agent: MarcEdit.5.2.Web.Client Disallow: / User-agent: MegaIndex.ru/2.0 Disallow: / User-agent: megite Disallow: / User-agent: milbot Disallow: / User-agent: mimas Disallow: / User-agent: mj12bot Disallow: / User-agent: mnogosearch Disallow: / User-agent: moget Disallow: / User-agent: mojeekbot Disallow: / User-agent: momspider Disallow: / User-agent: motor Disallow: / User-agent: msiecrawler Disallow: / User-agent: msnbot Disallow: / User-agent: MuscatFerre Disallow: / User-agent: myweb Disallow: / User-agent: NABOT Disallow: / User-agent: nagios Disallow: / User-agent: NaverBot Disallow: / User-agent: netcraft Disallow: / User-agent: netluchs Disallow: / User-agent: ng/2. Disallow: / User-agent: Ning Disallow: / User-agent: no_user_agent Disallow: / User-agent: nomad Disallow: / User-agent: nutch Disallow: / User-agent: ocelli Disallow: / User-agent: Offline Navigator Disallow: / User-agent: Offline+Navigator Disallow: / User-agent: onetszukaj Disallow: / User-agent: OurBrowser Disallow: / User-agent: parsijoo Disallow: / User-agent: pear.php.net Disallow: / User-agent: perman Disallow: / User-agent: PHP/ Disallow: / User-agent: pioneer Disallow: / User-agent: playmusic.com Disallow: / User-agent: playstarmusic.com Disallow: / User-agent: powermarks Disallow: / User-agent: psbot Disallow: / User-agent: PycURL Disallow: / User-agent: python Disallow: / User-agent: qihoobot Disallow: / User-agent: rambler Disallow: / User-agent: Readpaper Disallow: / User-agent: redalert|robozilla Disallow: / User-agent: RePEc.link.checker Disallow: / User-agent: robot Disallow: / User-agent: robots Disallow: / User-agent: RPT-HTTPClient/0.3-3E Disallow: / User-agent: rss Disallow: / User-agent: scan4mail Disallow: / User-agent: scientificcommons Disallow: / User-agent: scirus Disallow: / User-agent: scooter Disallow: / User-agent: seekbot Disallow: / User-agent: seznambot Disallow: / User-agent: shoutcast Disallow: / User-agent: SiteBot Disallow: / User-agent: slurp Disallow: / User-agent: sogou Disallow: / User-agent: speedy Disallow: / User-agent: spider Disallow: / User-agent: spiderman Disallow: / User-agent: spiderview Disallow: / User-agent: Steeler Disallow: / User-agent: Strider Disallow: / User-agent: sunrise Disallow: / User-agent: superbot Disallow: / User-agent: surveybot Disallow: / User-agent: tailrank Disallow: / User-agent: technoratibot Disallow: / User-agent: Teleport Pro Disallow: / User-agent: Teleport+Pro Disallow: / User-agent: Teoma Disallow: / User-agent: T-H-U-N-D-E-R-S-T-O-N-E Disallow: / User-agent: titan Disallow: / User-agent: turnitinbot Disallow: / User-agent: Twiceler Disallow: / User-agent: twiceler Disallow: / User-agent: ucsd Disallow: / User-agent: ultraseek Disallow: / User-agent: URL2File Disallow: / User-agent: urlaliasbuilder Disallow: / User-agent: urllib Disallow: / User-agent: validator Disallow: / User-agent: virus_detector Disallow: / User-agent: virus+detector Disallow: / User-agent: voila Disallow: / User-agent: w3c-checklink Disallow: / User-agent: Wanadoo Disallow: / User-agent: Web Downloader Disallow: / User-agent: Web+Downloader Disallow: / User-agent: WebCloner Disallow: / User-agent: webcollage Disallow: / User-agent: WebCopier Disallow: / User-agent: WebCorpusBuilder Disallow: / User-agent: Webinator Disallow: / User-agent: weblayers Disallow: / User-agent: Webmetrics Disallow: / User-agent: webmirror Disallow: / User-agent: webreaper Disallow: / User-agent: WebStripper Disallow: / User-agent: WebZIP Disallow: / User-agent: Wget Disallow: / User-agent: Win16 Disallow: / User-agent: wordpress Disallow: / User-agent: worm Disallow: / User-agent: www.gnip.com Disallow: / User-agent: WWW-Mechanize Disallow: / User-agent: xenu Disallow: / User-agent: Xenu Link Sleuth Disallow: / User-agent: Xenu Link+Sleuth Disallow: / User-agent: Xenu+Link Sleuth Disallow: / User-agent: Xenu+Link+Sleuth Disallow: / User-agent: y!j Disallow: / User-agent: yacy Disallow: / User-agent: yandex Disallow: / User-agent: YandexBot Disallow: / User-agent: Yeti Disallow: / User-agent: YisouSpider Disallow: / User-agent: yodaobot Disallow: / User-agent: zealbot Disallow: / User-agent: zeus Disallow: / User-agent: zyborg Disallow: /