# For domain: http://www.amusing.com #User-agent: BecomeBot #Disallow: / # All robots will spider the domain User-agent: * Disallow: User-agent: BecomeBot Crawl-Delay: 30 User-agent: * Disallow: /test/ Disallow: /forums/ Disallow: /functions/ Disallow: /cgi-bin/ Disallow: /images/