# search engine robot file. # Use this file to prevent areas # of your website from being indexed # by web robots. # Required for all crawlers to crawl your site ; User-agent: * # followed by Disallow: /whatever name User-agent: * Disallow: /grphcs/ Disallow: /images/ Disallow: /cgi-bin/ Disallow: /old/