############################### User-agent: * Disallow: /diaryofevents.php Disallow: /search.php User-agent: * Crawl-delay: 5 # multiple se spiders at same time to diaryofevents causing server overloads ###############################·