User-Agent: msnbot Crawl-Delay: 10 User-Agent: Slurp Crawl-Delay: 10 User-Agent: gigabot Disallow: / User-agent: ^Lydia Disallow: / # for all user agents User-agent: * # dont index these files Disallow: /scripts/ Disallow: /stat/ Disallow: /fm/ Disallow: *print_*.htm