# Eliminate crawling of full-text documents. # 7-30-2001 Neal Ensor User-agent: * Disallow: /disconps.html # 02/23/2007 NE removed by request for Jeff Given #Disallow: /servlets/purl/ #Disallow: /gpo/servlets/purl/ #Disallow: /doe/servlets/purl/ #Disallow: /etde/servlets/purl/ #Disallow: /dublincore/servlets/purl/ #Disallow: /infobridge/servlets/purl/ #Disallow: /dublincore/gpo/servlets/purl/ # WH 04/16/2007 Comment out innovation #Disallow: /innovation/research/ # 11/10/2006 # 01/30/2007 NE took out restrictions on ECD per Jeff Given #Disallow: /energycitations/product.biblio.jsp #Disallow: /energycitations/servlets/purl/ Disallow: /energycitations/searchresults.jsp Disallow: /bridge/searchresults.jsp Disallow: /energycitations/header.jsp # 07/13/2007 NE take out access to single-page searching Disallow: /bridge/singlepagesearch.jsp Disallow: /energycitations/singlepagesearch.jsp # 11/04/2008 WH Accomplishments pdfs are redirected Disallow: /accomplishments/pdf/ User-agent: Slurp Crawl-delay: 5