# Tell search engines to index these pages User-agent: * Sitemap: http://www.godo.com.au/dynamicsitemap.cfm Disallow: /assets Disallow: /xml User-agent: msnbot Crawl-delay: 10 User-agent: Slurp Crawl-delay: 5 User-agent: Googlebot Crawl-delay: 5