AI Crawlers #1204
Description
What would you like added?
Insert https://techcrunch.com/2025/03/27/open-source-devs-are-fighting-ai-crawlers-with-cleverness-and-vengeance/ into https://github.com/OWASP/wstg/blob/master/document/4-Web_Application_Security_Testing/01-Information_Gathering/03-Review_Webserver_Metafiles_for_Information_Leakage.md#robots
Would you like to be assigned to this issue?
Check the box if you will submit a PR to add the proposed content. Please read CONTRIBUTING.md.
- Assign me, please!