User-agent: googlebot Disallow: /bin/ Disallow: /datasources/ User-agent: slurp Disallow: /bin/ Disallow: /datasources/ User-agent: msnbot Disallow: /bin/ Disallow: /datasources/ User-agent: teoma Disallow: /bin/ Disallow: /datasources/ User-agent: WDG_SiteValidator Disallow: /bin/ Disallow: /datasources/ User-agent: Mediapartners-Google* Disallow: /bin/ Disallow: /datasources/ User-agent: Ocelli Disallow: /bin/ Disallow: /datasources/ Crawl-delay: 25 User-agent: twiceler Disallow: / User-agent: * Disallow: /