can anyone check if i've set up my robots.txt file properly? i'm not entirely sure how to do it but i do NOT want AI feasting on this site https://lapistoria.neocities.org/robots.txt
1 like
starry-knight
8 months ago
Yeah I think you did it right! That's pretty much how mine looks except I allowed Googlebot and DuckDuckbot so my site is indexed by search engines. I would also recommend making a sitemap.xml file if you allow those bots. It allows them to find and index your web pages more easily :3
1 like
i have been getting some trolls on the guestbook lately so i'm requiring email to sign it for a while
This'll be Retsu's character/song comments... Eventually...