# robots.txt for http://student.mit.edu/ User-agent: gsa-crawler Disallow: User-agent: * Disallow: /