User-agent: * Disallow: /Application.cfm Disallow: /error.cfm Disallow: /left_nav.cfm Disallow: /left_nav2.cfm Disallow: /lib Disallow: /temp.cfm Disallow: /cgi-bin Disallow: /nettracker Disallow: /old/ Disallow: /Connections/ Disallow: /content/ Disallow: /loc_lib/ Disallow: /scripts/ Disallow: /stats/ Disallow: /usage/ Disallow: /Multimedia/POTD/Photo_Gallery/themes/ Disallow: /Multimedia/POTD/Photo_Gallery/zen/ Disallow: /Multimedia/POTD/Photo_Gallery/trash/ #idiot-bot; generates tons of 404 and malformed urls User-agent: wget Disallow: / User-agent: ExtractorPro Disallow: /