User-agent: * Crawl-delay: 20 Request-rate: 1/20 Disallow: /mod/forum/search.php Disallow: /course/recent.php Disallow: /sites/ Disallow: /sites/*.* Disallow: /local/moodleorg/top/sites/ Disallow: /local/moodleorg/top/sites/*.* Disallow: /plugins/download.php Disallow: /plugins/download.php/*/*.* Disallow: /soft/* User-agent: Adsbot-Google Disallow: /*.*