# Allow "kongelig bibliotek" web crawler full access to site User-agent: KglBiblCrawl Disallow: # Deny all other spiders User-agent: * Disallow: /