# /robots.txt file for http://www.boeoegg.ch/ # have to allow Googlebot here since it will index # but not visit the page. I don't want it to be # indexed! User-agent: Googlebot User-agent: Ask Jeeves/Teoma Disallow: /analog Disallow: /App_Data Disallow: /aspnet_client Disallow: /bots Disallow: /botsv Disallow: /fpdb Disallow: /graphics Disallow: /htmlarea Disallow: /include Disallow: /intern Disallow: /picture_library Disallow: /publikation Disallow: /scripts Disallow: /sechselaeuten Disallow: /style Disallow: /webmin Disallow: /zuenfte Disallow: /zunftwesen Disallow: /zzz User-agent: * Disallow: * # common excludes for bad bots # They're banned via .htaccess, if they obey robots.txt they won't trigger an alarm User-agent: Art-Online Disallow: / User-agent: CherryPicker Disallow: / User-agent: Crescent Internet ToolPak Disallow: / User-agent: EmailCollector Disallow: / User-agent: EmailSiphon Disallow: / User-agent: EmailWolf Disallow: / User-agent: ExtractorPro Disallow: / User-agent: HTTPClient Disallow: / User-agent: Indy Library Disallow: / User-agent: Java Disallow: / User-agent: Microsoft URL Control Disallow: / User-agent: NexaBot Disallow: / User-agent: Nutch Disallow: / User-agent: VoilaBot Disallow: / User-agent: wavepluz Disallow: /