KEY TAKEAWAYS
- Scraping publicly available data is broadly defensible in the US after hiQ v. LinkedIn and Meta v. Bright Data — but "public" and "polite" are both load-bearing words.
- The legal risk has shifted from whether you scrape to what you scrape (personal data → GDPR/CCPA) and what you do with it (AI training → EU AI Act transparency).
- Since August 2025, EU AI Act rules require general-purpose AI providers to summarize training data — making source provenance documentation a purchasing requirement, not a nice-to-have.
- Buyer due diligence is simple: public data only, rate limits respected, PII scrubbed, provenance documented, ISO-certified processes. Get it in writing.
Is web scraping legal? The 2026 picture in one section
Web scraping is legal in most jurisdictions when it collects publicly available data without bypassing access controls — that principle has been repeatedly affirmed in US courts, most famously in hiQ Labs v. LinkedIn, where the Ninth Circuit held that scraping public profiles did not violate the Computer Fraud and Abuse Act (CFAA). What changed by 2026 is where the risk lives.
Ten years ago, the argument was about access: is an automated visitor "unauthorized"? Today that question is largely settled for public pages. The live questions are about content and use: personal data triggers privacy law, copyrighted text triggers IP questions, and AI training triggers the EU AI Act's transparency regime. A pricing feed of competitor SKUs sits at the low-risk end; a corpus of user profiles with names attached sits at the other.
What US courts have actually decided
The US case law that matters for buyers fits in three lines. hiQ v. LinkedIn (Ninth Circuit, 2019, reaffirmed 2022) established that accessing public data isn't "without authorization" under the CFAA — the anti-hacking statute doesn't apply to pages anyone can open. Van Buren v. United States (Supreme Court, 2021) narrowed the CFAA further, rejecting the idea that violating a use policy is a federal crime.
Then Meta v. Bright Data (N.D. Cal., 2024) addressed the contract angle: the court found that scraping public Facebook and Instagram data while logged out did not breach Meta's terms, because those terms bind account holders using their accounts — not visitors to public pages. It was a significant win for the data industry, though contract risk hasn't vanished: scraping behind a login, with an account that accepted terms, remains a genuinely different situation.
The pattern across a decade of US rulings: courts protect public data collection, and consistently punish circumvention — fake accounts, credential sharing, bypassing technical blocks after a cease-and-desist.
GDPR and privacy law: "public" doesn't mean "free to use"
European privacy law takes a different axis entirely. Under GDPR, personal data that someone posted publicly is still personal data — collecting it at scale requires a lawful basis, and regulators have acted on this. The Clearview AI enforcement wave made the point unmistakably: data protection authorities in France, Italy, Greece and the UK issued multi-million-euro fines for scraping faces and profiles, regardless of the data being "public."
For commercial data buyers, the practical rule is clean: the value is almost never in the personal data. A price-intelligence feed needs the SKU, price, seller name and timestamp — not the reviewer's real name. This is why our pipelines scrub PII at collection rather than promising to handle it later: reviews keep their text and star rating; usernames, emails and phone numbers never enter the dataset.
The Indian angle: DPDP Act
India's Digital Personal Data Protection Act (2023) follows a similar logic with one notable carve-out: it applies to digital personal data but exempts personal data "made publicly available by the data principal herself." Even so, downstream uses still carry obligations — and cross-border engagements are typically designed to the stricter GDPR standard anyway, so one pipeline satisfies both.
The EU AI Act changed what "compliant data" means
The EU AI Act entered into force in August 2024, and its obligations for general-purpose AI (GPAI) models became applicable in August 2025. Two provisions reshaped the training-data market: providers must maintain a policy to respect EU copyright law (including text-and-data-mining opt-outs), and they must publish a sufficiently detailed summary of training content.
The buyer-side consequence is concrete. If your model may ever serve the EU market, you need to answer "where did this data come from?" — per source, in writing. A crawl dump with no lineage can't answer that; a documented corpus can. It's the reason provenance documentation moved from an enterprise nicety to a standard line item in our AI & LLM training data engagements, and why one AI client's investors asked for it before their round closed (that case study is here).
The buyer's compliance checklist
You don't need to be a lawyer to buy web data responsibly. You need written answers to seven questions:
| Ask the vendor | The answer you want |
|---|---|
| What do you collect? | Publicly available data only — no logins, no paywalls, no access-control circumvention |
| How do you collect it? | Rate-limited crawling that respects platform stability; robots.txt considered per engagement |
| Is personal data included? | PII scrubbed at collection by default; documented exceptions only with a lawful basis |
| Can you document sources? | Per-record or per-source provenance, EU AI Act-ready for training data |
| What about copyright? | Facts (prices, availability, specs) over expression; TDM opt-outs respected for corpora |
| Security certifications? | ISO 27001 (security) and ISO 9001 (quality) or equivalent, with audit reports |
| Contractual protection? | NDA-first, clear warranties on collection methods, defined liability |
Want the compliance answers in writing?
Every engagement starts with an NDA and a documented collection methodology — before any data changes hands.
Where your use case sits on the risk spectrum
Not all scraping projects carry the same profile. In our engagements, use cases cluster into three bands:
Low risk — facts about products and prices. Competitor prices, stock status, menus, specs, delivery fees. No personal data, no creative expression, collected from public pages. This covers most price intelligence, availability tracking and MAP monitoring work.
Medium risk — user-generated content. Reviews and ratings involve copyrighted text and potential PII. Managed correctly (PII scrubbed, used for analysis rather than republication), this is routine — but it's where vendor discipline starts to matter.
Higher scrutiny — training corpora and anything behind login. AI training data needs the full provenance treatment post-AI Act. Data behind logins or paywalls needs case-by-case legal review, and a reputable vendor will sometimes tell you no — which is itself a signal you've found a reputable vendor.
The bottom line for 2026
Web scraping's legality question has matured from "is this allowed?" to "is this well-run?" Public data, collected politely, with personal information scrubbed and sources documented, supports thousands of legitimate businesses — including most of the price comparison, market research and AI products you already use. The failures that make headlines share a signature: circumvention, personal data at scale, or nothing in writing.
Buy accordingly: demand the documentation, and treat any vendor who waves the question away as your cheapest due-diligence test.
This article is general information, not legal advice. For decisions on a specific project, consult counsel in your jurisdiction. Key references: hiQ Labs v. LinkedIn (9th Cir.); Van Buren v. United States (2021); Meta Platforms v. Bright Data (N.D. Cal. 2024); Regulation (EU) 2024/1689 (AI Act), GPAI obligations applicable Aug 2025; GDPR; India DPDP Act 2023.
Frequently asked questions
In most jurisdictions, collecting data that is publicly visible without login is broadly defensible, following rulings like hiQ v. LinkedIn in the US. Legality still depends on how you collect (rate limits, no circumvention of access controls) and what you collect (personal data triggers GDPR and similar privacy laws).
Terms of service are a contract question, separate from criminal law. US courts have generally treated public-data scraping as outside the Computer Fraud and Abuse Act, but breach-of-contract claims remain possible where terms were actively accepted. Reputable providers manage this risk through collection design and legal review per engagement.
Since August 2025, providers of general-purpose AI models must publish a summary of training data and maintain a copyright policy under the EU AI Act's transparency rules. Practically, teams buying training corpora now need per-source provenance documentation — which is why our AI training data ships with it by default.
Publicly posted personal data is still personal data under GDPR. Using it requires a lawful basis, and regulators have fined companies for scraping profiles at scale. The safe pattern for commercial datasets is PII-scrubbing at collection: keep the product, price and review text; drop names, handles and contact details.
Both parties can carry exposure, which is why buyer due diligence matters. Ask vendors how they collect (public data only? rate limits respected?), what documentation they provide (provenance, PII policy), and whether they hold security certifications such as ISO 27001. A vendor who can't answer in writing is the red flag.