# /robots.txt file for http://webcrawler.com/

User-agent: *
Disallow: /cemdocs
Disallow: /IRTAD
Disallow: /Accidents
Disallow: /images
Disallow: /Search
Disallow: /Structure
Disallow: /