; $Id: ows_bots.txt 55 2007-07-18 15:27:16Z randomperson83 $ ; ; List of bots -- Pretty simple format: ; ; [string] \tabs [name of bot] ; ; Where string is the string in the agent-string to look for ; ; At this time, this list is not meant to be strictly authorative. In fact, its ; entirely possible that this list incorrectly lists things that aren't bots. ; If you find that this is the case, let me know and I'll fix it. ; ;"big" bots.. msnbot-media MSN Media msnbot MSN Googlebot-Image Google Images Googlebot Google Mediapartners-Google Google Adsense Yahoo! Slurp Yahoo! YahooFeedSeeker Yahoo! Feed Seeker Yahoo-Blogs/ Yahoo! Blogs Yahoo-MMCrawler/ Yahoo! MMCrawler YahooSeeker/ Yahoo! Mobile? Ask Jeeves/Teoma Ask.com Baiduspider Baidu Technoratibot/ Technorati archive.org_bot Archive.org ;other bots ia_archiver Alexa libwww-perl libwww-perl based crawler Nutch Nutch-based crawler ; random bots I got from my own website logs and other contributed bots AboutUsBot/ AboutUs BecomeBot/ Become Blogslive Blogslive BlogsNowBot BlogsNowBot BlogPulseLive BlogPulseLive boitho.com-dc/ Boitho ConveraCrawler/ ConveraCrawler del.icio.us-thumbnails del.icio.us e-SocietyRobot e-SocietyRobot Entireweb Entireweb Speedy Spider EmeraldShield.com Web Spider EmeraldShield Exabot Exabot FAST Enterprise Crawler FAST Enterprise Crawler FacebookFeedParser Facebook Notes Feedster Crawler/ Feedster genieBot genieBot Gigabot Gigabot gsa-crawler Google Search Appliance (not google) gsa-crawler-upgrade-config Google Search Appliance (not google) Girafabot Girafat GurujiBot/ GurujiBot heritrix/ Heritrix-based crawler holmes/ Holmes ichiro/ ichiro ICC-Crawler ICC-Crawler Interseek Interseek IRLbot IRL Crawler Jigsaw/ W3C CSS Validator Labrador/ Labrador larbin_ larbin LeapTag LeapTag LinksManager.com_bot LinksManager Megaglobe Crawler/ Megaglobe Crawler MyFamilyBot/ MyFamilyBot MojeekBot/ Mojeek Moreoverbot/ Moreover MJ12bot/ MJ12bot MSIECrawler MSIE Offline Favorites MSRBOT Microsoft Research Bot NetResearchServer/ NetResearchServer NextGenSearchBot NextGenSearchBot OGSearchSpider OG Search OmniExplorer_Bot/ OmniExplorer_Bot OutfoxBot/ Outfox Pingdom GIGRIB Pingdom pythonic-crawler pythonic-crawler psbot/ psbot QihooBot QihooBot Quantcastbot/ Quantcastbot SBIder SBIder semanticdiscovery/ Semantic Discovery SEOChat::Bot SEOChat Bot SeznamBot/ Seznam Shim-Crawler Shim-Crawler Snapbot Snap Shots SnapPreviewBot SnapPreviewBot Sogou web spider Sogou So-net RSS Crawler So-net RSS Crawler Sphere Scout Sphere Scout sproose/ Sproose SurveyBot/ SurveyBot UniversalFeedParser Universal Feed Parser based bot Touche Touche TridentSpider/ Trident Twiceler- Twiceler VadixBot VadixBot VoilaBot Voila voyager/ voyager VSynCrawler/ VSynCrawler wwwster/ wwwster W3C_Validator/ W3C Validator WebaltBot/ WebaltBot WebRankSpider/ WebRankSpider yacy yacy Yeti/ Yeti YodaoBot/ YodaoBot Yoriwa/ Yoriwa ZyBorg/ ZyBorg