midgard/zeus.ugent.be
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Merge pull request #369 from ZeusWPI/fix/disallow-robots

Browse source 
Allow less crawling
This commit is contained in:
redfast00 2020-01-29 01:46:26 +01:00 committed by GitHub
commit 0f939c09d7
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 15 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

18
content/robots.txt
View file

@ -1,4 +1,16 @@
User-agent: *
Allow: /
Disallow: /wiki/
Disallow: /slackintegrations
Disallow: /
Allow: /index.html
Allow: /$
Allow: /about/
Allow: /assets/images/
Allow: /blog/
Allow: /contact/
Allow: /events/
Allow: /feed.xml
Allow: /google6f2e77d0228abc35.html
Allow: /ical.ics
Allow: /projects/
Allow: /robots.txt
Allow: /sitemap.xml
Sitemap: https://zeus.gent/sitemap.xml