User-agent:*
Disallow:/products/
Disallow:/files/
User-agent: Slurp
Crawl-delay: 10