# # This file is to prevent the crawling and indexing of certain parts # of your site by web crawlers and spiders run by sites like Yahoo! # and Google. By telling these "robots" where not to go on your site, # you save bandwidth and server resources. # # This file will be ignored unless it is at the root of your host: # Used: http://example.com/robots.txt # Ignored: http://example.com/site/robots.txt # User-agent: * #Crawl-delay: 10 # Directories Disallow: /admin/ Disallow: /common/ Disallow: /fckeditor/ Disallow: /images/ Disallow: /include/ Disallow: /sites/ Disallow: /user/ # Files Disallow: /CheckLogin.php Disallow: /config.php Disallow: /default.php Disallow: /index.php Disallow: /Login.php Disallow: /notfound.htm Disallow: /robots.txt # Paths (clean URLs) #Disallow: /Home/ Allow: /index.php Allow: /default.php # Paths (no clean URLs)