prehnra/pre.hn
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat: Disable crawling for AI

Browse source
This commit is contained in:
Robert Prehn 2023-12-10 10:51:57 -06:00
parent 3682a28cb6
commit daa1abdca1
No known key found for this signature in database
1 changed files with 3 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
priv/static/robots.txt
View file

@ -1,3 +1,6 @@
User-agent: *
Allow: *
Sitemap: https://pre.hn/sitemap.xml.gz
User-agent: Google-Extended
Disallow: /