I didn't realise newer Neocities sites automatic have a robots.txt file to prevent them from getting snooped by ChatGPT and stuff, are you supposed to leave the # on the last line or not?
This configuration is very effective at blocking AI bots, but it has a major drawback: it also blocks several services that are not just AI-related. Consider exemptions.
![[tfpXE] avatar](/site_screenshots/15/25/tfpxe/index.html.50x50.webp)
![auberylis [dot] moe avatar](/site_screenshots/34/10/astrossoundhell/index.html.50x50.webp)
![xX [ TR/BZ ] Xx avatar](/site_screenshots/41/77/toribytez/index.html.50x50.webp)
![.:.[heather saturnia].:. avatar](/site_screenshots/11/46/heathersaturnia/index.html.50x50.webp)
https://caesar.wtf/robots.txt
This configuration is very effective at blocking AI bots, but it has a major drawback: it also blocks several services that are not just AI-related. Consider exemptions.
The last line is for making specific calls: