# All robots will spider the domain User-agent: * Disallow: /assets/images/ Disallow: /assets/menu/ Disallow: /assets/css/ Disallow: /assets/js/ Disallow: /counter/ Disallow: /mail/ Disallow: /restricted/ Disallow: /errordocs/ Disallow: /glossary/