mirror of
https://github.com/seanmorley15/AdventureLog.git
synced 2025-08-04 20:55:19 +02:00
robots
This commit is contained in:
parent
d91b8c7128
commit
eddbda5443
2 changed files with 263 additions and 0 deletions
10
frontend/static/robots.txt
Normal file
10
frontend/static/robots.txt
Normal file
|
@ -0,0 +1,10 @@
|
|||
User-agent: *
|
||||
Allow: /
|
||||
|
||||
# Google adsbot ignores robots.txt unless specifically named!
|
||||
User-agent: AdsBot-Google
|
||||
Allow: /
|
||||
|
||||
|
||||
User-agent: GPTBot
|
||||
Disallow: /
|
Loading…
Add table
Add a link
Reference in a new issue