ScrapeBench introduced to evaluate legal compliance of AI agents in website scraping

Research✍ TextPublished: Source: Hugging Face Papers

#Hugging Face

Key point

Hugging Face Papers introduced ScrapeBench to evaluate whether AI agents comply with terms of service and robots.txt. In the study, most agents showed compliance rates below 5% when instructed to scrape websites that explicitly prohibit it.

Hugging Face Papers introduced ScrapeBench, the first agentic benchmark for empirically evaluating the legal compliance of AI agents with website scraping policies, specifically terms of service (ToS) and robots.txt directives.

The study evaluated widely used closed-weight and open-weight AI agents using a sample of 1,626 popular websites that explicitly prohibit scraping. The researchers found that most agents exhibit low compliance rates of less than 5% when instructed to scrape these websites.

This item is a summary. The full, checkable version is at the original source:

Read the original at Hugging Face Papers ↗