# Robots.txt file from http://www.beitunren.com # All robots will spider the domain User-agent: * #Sitemap Sitemap:http://www.beitunren.com/sitemap.xml #Directories Disallow:/ADMIN/ Disallow:/SCRIPT/ Disallow:/PLUGIN/ Disallow:/FUNCTION/ Disallow:/UPDATE/ Disallow:/ARTICLE-IMAGE/ Disallow:/DATA/ Disallow:/CACHE/ Disallow:/STYLE/ Disallow:/TEMPLATE/ Disallow:/CSS/ Disallow:/LANGUAGE/ Disallow:/ALEXA/ #Files Disallow:/httperrors/404.htm Disallow:/login.asp Disallow:/cmd.asp? Disallow:/search.asp Disallow:/atom.xml Disallow:/view.asp? Disallow:/catalog.asp?page=1 Disallow:/catalog.asp?cate=1 Disallow:/catalog.asp?cate=2 Disallow:/catalog.asp?cate=3 Disallow:/catalog.asp?cate=4 Disallow:/catalog.asp?cate=5 Disallow:/catalog.asp?cate=6 Disallow:/catalog.asp?cate=7 Disallow:/catalog.asp?cate=8 Disallow:/catalog.asp?cate=9 Disallow:/catalog.asp?cate=10 Disallow:/catalog.asp?cate=11 Disallow:/catalog.asp?cate=12