zeus.ugent.be/content/robots.txt

User-agent: *
Disallow: /
Allow: /about/
Disallow: /about/sponsorende-leden/
Allow: /contact/
Allow: /events/
Allow: /projects/
allow robots to do all the things 2017-05-29 12:44:07 +00:00			`User-agent: *`
Allow less crawling 2020-01-16 08:02:52 +00:00			`Disallow: /`
			`Allow: /about/`
			`Disallow: /about/sponsorende-leden/`
			`Allow: /contact/`
			`Allow: /events/`
			`Allow: /projects/`