# $Id$ # robots.txt RFC: http://www.robotstxt.org/wc/norobots-rfc.html # Validated by http://www.searchengineworld.com/cgi-bin/robotcheck.cgi User-agent: * Disallow: /admin/ Disallow: /cert/ Disallow: /data/ Disallow: /demo/ Disallow: /images/ Disallow: /js/ Disallow: /sytle/ Disallow: /test/ Disallow: /upload/ #Disallow: /nettv/ Disallow: /WEB-INF/ Disallow: /ad/ Disallow: /c/ Disallow: /p/ User-agent: Slurp Crawl-delay: 7 User-agent: iaskspider Disallow: / User-agent: Baiduspider Disallow: /query/ Disallow: /bt/download/ User-agent: turnitinbot Disallow: / User-Agent: YodaoBot Disallow: / User-Agent: Gigabot Disallow: / User-Agent: Yahoo! Slurp Disallow: / User-Agent: CCBot Disallow: /