Perplexity, recently acclaimed for its revolutionary capabilities, finds itself at the center of a turmoil. Accusations of unauthorized use of content are emerging, raised by Cloudflare in a striking report. The contested practice of “scraping” raises questions about the ethics of search engines. The bots of Perplexity, accused of circumventing the restrictions imposed by websites, see their actions questioned. The possibility of strict regulation on data access could transform the digital landscape.
Serious accusations of circumventing restrictions
Less than a month after the launch of its Comet project, Perplexity, an AI-based browser accessible only via the Perplexity Max subscription, is facing serious accusations. Cloudflare, the online service for accelerating and protecting websites, recently published a report that calls into question Perplexity’s scraping practices.
The controversial scraping by bots
Since its inception, Perplexity has established itself as a leading player in the AI sector, thanks to its ability to conduct in-depth web searches. However, this functionality raises questions about compliance with the rules imposed by website owners. The latter often stipulate that they do not wish to be visited by crawlers, i.e., the exploration bots used by search engines.
Websites can indicate these preferences in a file named robots.txt, or through WAF-like filters (Web Application Firewall). According to the Cloudflare report, Perplexity allegedly circumvented these restrictions, accessing sensitive information while masking its activity in case of blockage.
Cloudflare’s findings
Cloudflare cites anomalies in the behavior of PerplexityBot and Perplexity-User bots. This report raises questions about scraping practices, confirming that these bots may have accessed data using techniques to conceal their identity. Screenshots support these allegations, showing potentially compromised data.
Perplexity’s response
Perplexity’s reaction to these accusations is immediate. Jesse Dwyer, the company spokesperson, characterized Cloudflare’s report as just a publicity stunt. He also mentioned that this study contained several inconsistencies that could invalidate the presented conclusions.
Concerns about Cloudflare’s business model
In parallel, Cloudflare has begun to consider introducing a business model where AI would be required to pay to scrape websites, named Pay per Crawl. The danger of such a system could undermine access to information on the Internet, thus raising ethical questions about the openness of the web.
An ongoing debate
The controversy between these two companies raises a heated debate in the field of artificial intelligence and digital ethics. The implications of these accusations could influence the future of relationships between AIs and websites, with both commercial and legal stakes.
To learn more about the topic, articles such as the one on the music legend or recent innovations like ChatGPT’s research tool provide context to this issue.
Common Frequently Asked Questions
What accusations have been made against Perplexity by Cloudflare?
Cloudflare accuses Perplexity of circumventing “robots.txt” regulations and scraping content on websites without permission, using techniques to hide their activity.
How does Perplexity conduct its web searches?
Perplexity uses bots called “crawlers” to visit web pages and extract content. This requires a process known as “scraping”.
What does the term “scraping” mean in the context of Perplexity?
“Scraping” refers to the method that Perplexity’s bots use to collect information on the web by accessing different sites.
What is Perplexity’s response to Cloudflare’s accusations?
Perplexity labeled Cloudflare’s report as a “publicity stunt” and stated that the study contained many inconsistencies and did not reflect their actual operation.
What measures can websites take to prevent scraping by bots like those from Perplexity?
Websites can use a “robots.txt” file to indicate to crawlers which pages they do not wish to be accessed and apply WAF filters to block unwanted traffic.
What evidence has Cloudflare presented to support its accusations against Perplexity?
Cloudflare has provided screenshots and data showing that Perplexity’s bots accessed websites despite explicit prohibitions, as well as indications of modifying ASN numbers to conceal their activities.
Could the controversy surrounding Perplexity impact the company?
The controversy could affect Perplexity’s reputation and its relationship with the affected websites, as well as its subscription model based on artificial intelligence.
