AI Search Engine Perplexity Accused of Evading Internet Privacy Norms

AI search engine Perplexity is facing accusations of violating web norms by using stealth tactics to bypass restrictions set by websites through robots.txt files and firewalls. Cloudflare's investigation revealed that when Perplexity was blocked, it deployed a stealth bot employing various methods, including rotating IP addresses, to continue scraping content from tens of thousands of domains. This behavior raises significant concerns regarding adherence to the Robots Exclusion Protocol, which is designed to protect website owners' rights and data integrity. For healthcare technology professionals, this situation underscores the importance of robust data governance and the potential vulnerabilities associated with automated content scraping.

Opens on Ars Technica