# Tells Scanning Robots Where They Are And Are Not Welcome # # User-agent: can also specify by name; "*" is for everyone # Disallow: disallow if this matches first part of requested path # # For now disallow all we can modify this as needed to allow certain crawlers. # User-agent: * Disallow: /_private Disallow: /_vti_bin Disallow: /_vti_cnf Disallow: /_vti_log Disallow: /_vti_pvt Disallow: /_vti_txt Disallow: /acidwww Disallow: /cgi Disallow: /frankwww Disallow: /images Disallow: /php_uploads Disallow: /scottwww Disallow: /stats Disallow: /WEB-INF Disallow: /filez