Felix Van der Jeugt
|
3d311ac8e6
|
Allow crawling the sponsoring members page
|
2020-01-27 21:28:03 +01:00 |
|
|
Felix Van der Jeugt
|
e4a58127f0
|
Also allow plain home page
|
2020-01-17 19:31:11 +01:00 |
|
|
Felix Van der Jeugt
|
75c8e6af92
|
Allow crawling the home page
|
2020-01-16 15:50:17 +01:00 |
|
|
Felix Van der Jeugt
|
5c5a2275f2
|
Use sitemap to allow more pages
|
2020-01-16 09:24:05 +01:00 |
|
|
Felix Van der Jeugt
|
785c927834
|
Allow less crawling
|
2020-01-16 09:02:52 +01:00 |
|
lorin
|
f9c0fe8e1a
|
Update robots.txt
|
2018-03-27 22:24:52 +02:00 |
|
Rien Maertens
|
c9b18b96df
|
Disallow /wiki/ in robots.txt
|
2017-12-17 11:35:51 +01:00 |
|
|
Lorin Werthen
|
5227e098a5
|
allow robots to do all the things
|
2017-05-29 14:44:07 +02:00 |
|