The New York Times Sued Brave. The Real Story Is a Crawler With No Name (2026)

The most important sentence in Monday's lawsuits against Brave is a shrug. Lawyers for The New York Times and three Tribune newspapers filed 141 pages between them, and partway through they concede they can't say what the defendant's crawler is called: "Brave, however, has never disclosed the name of its 'bot.'"

On 2 November 2026, The New York Times and the publishers of the New York Daily News, Chicago Tribune and Orlando Sentinel sued Brave Software in the Southern District of New York. They allege Brave copied their articles into its 30-billion-page Brave Index with an unnamed crawler and sold access to AI companies through its Search API.

The complaints name Anthropic, Perplexity, Mistral AI, xAI and Cohere as buyers; none of them is a defendant. For a website owner, the bot's missing name matters more than any damages figure, because every opt-out in 2026 depends on a crawler identifying itself. The Times can answer that with a lawsuit. You probably can't, so after the allegations this piece turns into an audit you can run this afternoon.

Last updated: 3 November 2026. Everything below about Brave is an allegation in a complaint, not a finding by a court, and fair use for AI training is undecided. This isn't legal advice; if you have real exposure, talk to a lawyer. The technical advice is mine, and every number links to the source I read it in.

One disclosure. You're reading this on a directory's blog. SaaSCity is a startup directory drawn as a live city map, and one of my conclusions is that every founder needs at least one distribution channel that runs on a contract rather than a crawl. A directory listing is that kind of channel, so I have a stake in where this lands.

What the Times and the Tribune papers allege

The New York Times and three Tribune papers allege that Brave Software copied their journalism into the Brave Index, served it back in its own AI answers and sold it to AI companies through the Brave Search API, all without a licence.

The Times filed first, on Monday 2 November 2026, in the US District Court for the Southern District of New York, case 26-cv-9549, according to PPC Land's breakdown of both filings. Hours later the publishers of the Daily News, Chicago Tribune and Orlando Sentinel, owned by Alden Global Capital through MediaNews Group and Tribune Publishing according to TheNextWeb, filed case 26-cv-9554. TheWrap counted 72 pages in the first complaint and 69 in the second. Steven Lieberman of Rothwell, Figg, Ernst & Manbeck signed both.

	The New York Times v. Brave Software	Daily News et al. v. Brave Software
Case number	26-cv-9549	26-cv-9554
Length	72 pages	69 pages
Counts	Direct, contributory and vicarious copyright infringement; DMCA anti-circumvention (17 U.S.C. 1201); unjust enrichment under New York law	The same five counts
Relief sought	Damages and an injunction	Statutory and actual damages, disgorgement of profits, a permanent injunction, a jury trial
Scale claimed	Not given in coverage	More than two million registered articles

TechCrunch quotes the Times calling Brave a "supplier, dealer, and distributor of stolen copyrighted content" that "acts both like a burglar that misappropriates the content of publishers" and like an AI "fence." Lieberman gave TheWrap the theory in one line: "Buying stolen goods secondhand doesn't make them legitimate."

Specific examples back the language up. According to TheWrap, Ask Brave, the AI answer feature in Brave Search, produced more than 1,200 words of a Chicago Tribune story on tax breaks for data centers and nearly 600 words of a Daily News story on Central Park horse carriages. PPC Land reports a starker case: Brave's LLM Context API returned 1,291 of the 1,295 words of the Times piece "Mixed Human Feelings After a Day at a Park Full of Robots." That is the whole article minus four words, delivered by the product Brave sells to AI developers.

The suits also target a "Paywall blocker" setting, off by default according to PiunikaWeb, and Brave's practice of blocking publishers' ads while selling its own. The Times says it wrote to CEO Brendan Eich on 11 August 2025 and 13 April 2026 without a reply; spokesperson Graham James told The Verge, as PiunikaWeb quotes him, that Brave "flouted established copyright law and ignored repeated requests by The Times to stop its infringing conduct." Brave, founded in 2015 and backed by roughly $250 million per TechCrunch, didn't respond to requests for comment, and I found no statement by the next afternoon.

A 30-billion-page index and a crawler with no name

The Brave Index is the independent web index behind Brave Search, more than 30 billion pages according to both complaints. The publishers say they couldn't keep their articles out of it because Brave's crawler never published a name they could block.

robots.txt runs on names. Each group starts with a User-agent line naming a crawler, followed by the paths it may fetch, and only the crawler itself enforces it. You can write a rule for GPTBot because OpenAI publishes the token. For a crawler that looks like an ordinary browser, the only rule left is the catch-all User-agent: * group, which blocks every compliant crawler, Google included.

Brave says so itself. Its Brave Search Crawler help page, as I read it on 3 November 2026: "The Brave Search crawler does not advertise a differentiated user agent because we must avoid discrimination from websites that allow only Google to crawl them." The next sentence is its opt-out: "if a domain or page is not crawlable by Googlebot, then Brave Search's bot will not crawl it either."

So Brave honours robots.txt, but only the rules you wrote for Google, and the one line that stops its crawler also stops Googlebot. Few businesses can afford that line. The same page adds that "robots.txt is not used to prevent a page from being indexed"; delisting takes a noindex tag, which drops the page from Google too.

The complaints say Brave crawled "without identifying itself or providing website owners with any mechanism to block its crawler." News Corp alleged the same in its countersuit of 21 July 2026, reported by TheWrap: "Brave masks its web crawlers such that publishers cannot detect or reliably block them." Third-party bot lists still publish a "Bravebot" token, but Brave's own page names none, so a Bravebot rule is a guess.

Five AI companies, zero defendants

Both complaints name Anthropic, Perplexity, Mistral AI, xAI and Cohere as customers of the Brave Index without suing any of them, which shows the strategy: sue the supplier, and every buyer downstream feels it.

The complaints cite a Brave blog post of 2 November 2023 saying "Anthropic used Brave's Search API to train their latest model, Claude 2," and TheNextWeb quotes Brave marketing that lists the API's common uses as "in training and grounding several of the Web's biggest large language models." Two of the five counts, contributory and vicarious infringement, concern someone else's infringement by definition: Brave is accused of enabling what its customers did with the copies and profiting from it.

The Times already has separate cases against OpenAI and Microsoft, filed on 27 December 2023 and still without a fair-use ruling as of 29 October 2026, per AI Vortex's case tracker, and against Perplexity, filed on 5 December 2025 per TechCrunch.

If your product grounds AI answers in a third-party search API, you sit where those five sit. Brave's FAQ answers "Can I store results or use them to train models?" with "Yes, with a plan that includes storage rights." Read your own vendor's version of that clause before someone else's lawsuit reaches discovery.

Brave's side of it, in Brave's own words

Brave's position, from its filings and product pages, is that indexing the public web is what every search engine does, that doing so is fair use, and that its independent index is an alternative to Google and Bing.

Brave argued this before anyone sued it. In March 2025, after a News Corp cease-and-desist, it asked a federal court in San Francisco to declare its indexing and AI summaries lawful, The Register reported, calling indexing to run a search engine "fair use."

Every search engine copies pages to build an index, Google included. TechCrunch's July 2026 report on Cloudflare's crawler policy notes that Googlebot crawls for Search, AI Overviews and AI Mode included, and Cloudflare says the "world's largest search engine" sees about "2x more information" than other AI companies. Brave's nameless user agent has a competition logic too: if sites allow Googlebot and block everyone else, a new engine that announces itself never builds an index. And Brave does follow an opt-out, namely Google's.

The marketing is harder to defend. On 3 November 2026 the Brave Search API page said "Trusted by the world's top AI labs (including ones we can't name)," listed AWS, Cohere, Mistral AI, Snowflake, You.com, Firecrawl and Kagi, and claimed "100s of millions" of daily queries and "150 billion" answered. Under the heading "No scraping. No fallback." it says "Our index is our own: crawled, ranked, and served end to end." Prices start at "Up to $20 in free calls every month."

"No scraping" holds in the sense Brave means: the API doesn't resell Google or Bing results. It says nothing about your page once it sits in that index and is sold, storage rights included, to a lab that trains on it, and that gap is what both lawsuits are about. No court has ruled on fair use for AI training in any of the Times' cases, so anyone calling this settled is guessing.

What a copied article costs a SaaS founder

A copied article costs a SaaS founder the clicks it was written to earn, and those clicks are how a content budget pays back.

Take made-up but ordinary numbers. A freelancer charges $300 a post, you publish 40 a year, and that's $12,000. A good post earns 1,000 Google visits a year and 2% of visitors start a trial: 20 trials per post. Now the question gets answered inside an AI tool that sends 95.7% less referral traffic than Google, a figure from a study the Tribune papers cite, per TheNextWeb, which doesn't name the study. A thousand visits become about 43, and 20 trials become fewer than one. The post still cost $300.

Three asymmetries follow. The Times can fund years of litigation, and its OpenAI case has run since December 2023; you can't, so your protection has to be defaults you set once and recheck. Your bargaining power sits in channels that are agreements, like a directory listing with a dofollow link and tagged referrals, while a page in a 30-billion-page index is stock on someone else's shelf. And you don't have to choose between being cited by AI and protecting your content: let search crawlers in, keep training crawlers out, and accept that a block today doesn't un-train a model that read you last year.

The 2026 opt-out stack, and exactly where it breaks

The 2026 opt-out stack has four mechanisms: named robots.txt tokens, Cloudflare's crawler categories, RSL licence files and the IETF's shared training token. Each one works only on crawlers that identify themselves or choose to listen.

Mechanism	Who enforces it	What it covers	What it misses
robots.txt per-vendor tokens	The crawler, voluntarily	Any bot whose operator publishes a token (GPTBot, ClaudeBot, Google-Extended and others)	Unnamed crawlers; user-triggered fetchers that may skip robots.txt; impostors
Cloudflare Search, Agent and Training categories	Cloudflare, at the edge	Bots Cloudflare has identified and classified, on every plan	Crawlers that pass as ordinary browsers; mixed-use bots get the strictest rule, Googlebot included
RSL 1.0 licence file	Nobody yet: terms are stated, not enforced	Any crawler that reads license.xml and chooses to comply	AI companies that haven't adopted it; Digiday reported in November 2025 that none had yet complied
IETF __usage_ai-training token	Each AI company that adopts it	Training use by any participating provider, whatever its crawler is called	Crawling itself; anyone who hasn't adopted it; Google and Bing until 2027

Named robots.txt tokens

A robots.txt token is the name a vendor publishes so you can address its crawler. OpenAI's crawler documentation says "Each setting is independent of the others," so you can allow OAI-SearchBot for ChatGPT search and disallow GPTBot for training. The table lists each token as its vendor documents it.

Job	Token	Documented by	What the vendor says about it
Training	GPTBot	OpenAI	Disallowing signals content shouldn't be used for training
Training	ClaudeBot	Anthropic	Honours robots.txt; a block excludes future material from training
Training and Gemini grounding	Google-Extended	Google	A control token, not a separate crawler; no effect on Google Search
Training	Applebot-Extended	Apple	Doesn't crawl; governs use of Applebot data; Apple search unaffected
Open training dataset	CCBot	Common Crawl	Block in robots.txt; warns of crawlers faking its name
Training	Meta-ExternalAgent	Meta	robots.txt applies
Search	OAI-SearchBot	OpenAI	Opting out removes you from ChatGPT search answers
Search	Claude-SearchBot	Anthropic	A block stops indexing for Claude's search
Search	PerplexityBot	Perplexity	"Not used to crawl content for AI foundation models"
Search	Meta-WebIndexer	Meta	Improves Meta AI search results
User-triggered	ChatGPT-User	OpenAI	"robots.txt rules may not apply"
User-triggered	Claude-User	Anthropic	A block stops retrieval for user queries
User-triggered	Perplexity-User	Perplexity	"Generally ignores robots.txt rules"
User-triggered	Meta-ExternalFetcher	Meta	"May bypass robots.txt rules"

Adoption is real but partial. AI Crawler Check read 4,962 robots.txt files from the Tranco top 10,000 on 5 October 2026: 15.4% blocked CCBot, 14.3% GPTBot and 13.4% ClaudeBot, but only 7.4% blocked OAI-SearchBot. It's a small vendor study of robots.txt alone, and it still shows more sites refusing training than search. The weak spot is the bottom of the table, where vendors say robots.txt may not bind user-triggered fetchers, plus every crawler that isn't in the table at all.

Cloudflare's crawler categories

Cloudflare's report of 1 July 2026, Content Independence Day, one year on, said 52% of crawler requests were for AI training in June 2026, up from 22% in spring 2025. The same day its bot changelog split AI traffic into "Search, Agent, and Training," and said that from 15 September 2026 new domains get Training and Agent bots blocked on pages that display ads, with Search allowed. The Decoder confirmed the controls reach the Free plan, and TechCrunch reported the defaults also cover existing free accounts.

A multi-purpose crawler is a bot that collects for search and AI training under one name, and Cloudflare's changelog says those "will be affected by the new defaults to block Training." The Decoder names Googlebot as the example. Cloudflare CEO Matthew Prince wants to "encourage mixed-use crawlers to separate out search from agent use and training"; until Google splits its crawler, a Training block at your edge can refuse Googlebot while robots.txt looks fine.

For sites that want paying instead of blocking, Pay Per Crawl answers unpaid bots with "HTTP 402 Payment Required," with Cloudflare as merchant of record, and the AI Crawl Control docs still label it "(private beta)." Our piece on Cloudflare's Web Search API for agents covers the buyer side. All of this applies to bots Cloudflare has identified; a crawler that passes as a browser is left to bot detection, which is probabilistic.

RSL 1.0 licence files

RSL (Really Simple Licensing) is an open standard that turns the yes or no of robots.txt into licence terms. RSL 1.0 was published on 10 December 2025 with Yahoo, Ziff Davis and O'Reilly Media as collaborators and more than 1,500 supporting organisations, including the Associated Press, The Guardian and Cloudflare. A License: line in robots.txt points to a license.xml file that sets terms per path, as the RSL homepage shows. Nobody has to obey it, though: Digiday reported on 26 November 2025 that AI companies had yet to comply.

The IETF training token

The __usage_ai-training token is a shared robots.txt policy token that states a training preference to every AI provider that honours it, whatever its crawler is called. Kevin Kelley of Anthropic defined it in draft-kelley-aipref-training-preference-signal, published on 30 October 2026.

Google's John Mueller wrote the same day on the Search Central blog that Google aims to support it "in 2027" and will treat a shared opt-out as a Google-Extended disallow. Coywolf reports that OpenAI will implement it "in the coming months," that Bing will honour it from February 2027, and that Anthropic alone uses it today. The draft says the token "is not a directive for crawling behavior," and Datatracker gives it "no formal standing." It removes the need to know a crawler's name, but the company behind the crawler still has to choose to listen.

Should you block AI training crawlers and still show up in AI answers?

Yes, for most SaaS sites. Block training crawlers, allow search and answer crawlers, and you keep the chance of being cited without handing your archive to the next training run.

Anthropic's crawler page, dated 7 April 2026, separates ClaudeBot, where a block "signals that the site's future materials should be excluded" from training, from Claude-SearchBot and Claude-User, which affect Claude's search results. Perplexity says PerplexityBot "is not used to crawl content for AI foundation models," and Apple says pages that disallow Applebot-Extended "can still be included in search results." Note the words "future materials": none of these pages offers to remove what a model already learned.

Google-Extended is the judgement call. It covers Gemini training and grounding in Gemini Apps, and Google's 30 October post says sites that allow it "are eligible to appear in responses" there, so blocking it may cost you Gemini answers while leaving Google Search alone. A starting file for a SaaS that wants citations but not training:

# Training crawlers: no
User-agent: GPTBot
User-agent: ClaudeBot
User-agent: Google-Extended
User-agent: Applebot-Extended
User-agent: CCBot
User-agent: Meta-ExternalAgent
Disallow: /

# Shared training preference (IETF draft, October 2026)
User-agent: __usage_ai-training
Disallow: /

# Search and answer crawlers: yes
User-agent: OAI-SearchBot
User-agent: Claude-SearchBot
User-agent: PerplexityBot
Allow: /

Our GEO playbook covers how to get quoted by the crawlers you let in.

Does blocking GPTBot or CCBot hurt your Google rankings?

No. Google Search doesn't read rules written for OpenAI's GPTBot or Common Crawl's CCBot, and Google's documentation says even its own Google-Extended token "does not impact a site's inclusion in Google Search nor is it used as a ranking signal."

The risk sits in your CDN. A Cloudflare Training block that treats Googlebot as multi-purpose can refuse Google on pages your robots.txt allows, so the file looks fine while search traffic drops. And by Brave's own account, the only robots.txt rule that keeps its crawler out is a Googlebot block, which no SaaS can afford to write.

How to audit your own site in an afternoon

An AI crawler audit takes an afternoon with your server logs, robots.txt, CDN dashboard and a terminal. Work through it in this order.

1. Inventory what crawls you. Pull 30 days of user agents from your server logs or Cloudflare's AI Crawl Control dashboard. List every bot by name and request count, and flag anything that claims to be a browser but fetches like a crawler.
2. Check which tokens your robots.txt names. List every User-agent line in yoursite.com/robots.txt and compare it with the tokens OpenAI, Anthropic, Perplexity, Google, Apple, Common Crawl and Meta document. Any token you don't name falls under your User-agent: * group, which usually means allowed.
3. Split training crawlers from search and answer crawlers. Disallow the training tokens you don't want, such as GPTBot, ClaudeBot, Google-Extended, Applebot-Extended, CCBot and Meta-ExternalAgent, and allow the search crawlers that cite you, such as OAI-SearchBot, Claude-SearchBot and PerplexityBot. Blocking training stops future collection only.
4. Check the CDN layer as well as robots.txt. robots.txt only asks; your CDN or firewall decides. Compare Cloudflare's Search, Agent and Training settings and any custom bot or WAF rules with your robots.txt, because an edge rule can block a crawler your file allows.
5. Check the Googlebot mixed-use trap. If your domain joined Cloudflare on or after 15 September 2026, or is on the Free plan, Training may be blocked by default on pages with ads, and that block also catches multi-purpose crawlers such as Googlebot. Confirm Googlebot gets a 200 in Search Console's URL Inspection tool and in your logs.
6. Publish an RSL licence if you would rather license than block. Write a license.xml following the RSL 1.0 specification, set terms per path, add a License: line to robots.txt pointing at it, and check it with the validator on rslstandard.org. Few AI companies honour RSL yet; you are putting your terms on record.
7. Verify with a real request and read the status code. Request your main URLs with curl using each crawler's user agent string and read the status code: 200 is allowed, 403 is blocked at the edge, 402 is a pay-per-crawl wall. CDNs often verify big crawlers by IP address, so your logs stay the final word.
8. Write down the decision and the date. Record which tokens you allowed or blocked, what the CDN does and today's date. Vendors add bots, Cloudflare changes defaults, and Google and Bing plan to honour the shared training token in 2027, so rerun the audit every quarter.

If you only have time for one step, make it step 5, the only one where a mistake costs you search traffic.

What to do if you cannot sue anyone

A founder who will never file a 72-page complaint protects their content with defaults set once and rechecked, assets a crawler can't copy, and distribution that comes from an agreement instead of an index.

License where it's free: an RSL file takes an hour and puts your terms on record. Keep the moat out of plain text, too. Your blog is the most copyable thing you own, and if a competitor could rebuild your value from your public posts, so can an AI answer. OpenAI's move to training on a cached web also means what a model knows about you gets fixed at snapshot time.

Then own one channel that's a contract. Rented platforms change terms overnight, as Reddit did when it killed RSS and its public API, and an index takes without asking. A directory listing, newsletter sponsorship or partner page is an agreement with someone you can name; our post on the SEO benefits of directory listings covers which ones still pass value.

The plug, labelled as one. SaaSCity is a gamified startup directory where a human editor reviews every listing. Each product gets a free permanent listing page, and a paid plan adds a building on a live city map. Add the SaaSCity badge to your site and you get a dofollow backlink and a Monday launch slot. Quick Pass ($19.99) goes live within 24 hours, Premium ($39.99) adds a written launch post with three dofollow links, and Max ($99.99) adds 30 days of billboard, ticker and homepage pin. The domain was rated DR 70 at our latest Ahrefs refresh (November 3, 2026). What you get is a deal you can inspect: one page, one link and referral clicks you can count.

What happens next

Brave has to answer both complaints. If either case survives a motion to dismiss, discovery is where the crawler's name and the five customers' contracts could reach the record.

In discovery, publishers who couldn't block an unnamed crawler can ask for its logs, user-agent strings and API contracts, and the five AI companies become possible sources of documents. On fair use, Judge Sidney Stein is weighing summary judgment in the OpenAI case, and AI Vortex, citing Axios, expects him to decide in 2027 whether it goes to trial. Brave also hasn't said whether it will honour the shared training token; if it did, it could solve the publishers' practical problem without ever naming its crawler.

Questions people are asking about the Brave lawsuit

What is the New York Times lawsuit against Brave about? The Times and three Tribune papers allege Brave copied their articles into its 30-billion-page index with an unnamed crawler and sold access to AI companies. Both suits were filed on 2 November 2026; nothing has been decided.

What claims did the publishers bring? Direct, contributory and vicarious copyright infringement, DMCA anti-circumvention and unjust enrichment under New York law.

What is the Brave Index and the Brave Search API? The index behind Brave Search, sold to developers through the Search API, which allows training "with a plan that includes storage rights."

Why can't site owners block Brave's crawler? It publishes no user-agent token and follows Googlebot's rules, so the only robots.txt block that stops Brave also stops Google.

Are Anthropic, Perplexity, Mistral AI, xAI and Cohere being sued? Not in these cases. They're named as customers; Brave is the only defendant.

How do I block AI crawlers in 2026? Disallow the vendor-documented training tokens in robots.txt, set Cloudflare's Training control if you use it, and check your logs.

How do I license content to AI companies? Publish an RSL 1.0 licence file and point to it from robots.txt. Expect little compliance yet.

What is the IETF AI training token? __usage_ai-training, a shared robots.txt token from an Anthropic-authored draft of 30 October 2026. Google and Bing plan support in 2027.

Has Brave responded? Not as of 3 November 2026. Its earlier position, from its 2025 suit against News Corp, is that indexing is fair use.

Why does a directory listing matter if crawlers can copy my content? It's an agreement you can audit: a reviewed page, a backlink and referral clicks you can count.

Crawled or contracted

The web is splitting into two internets. In the crawled one, your pages sit in someone's index and get sold by the query to whoever is buying, whether or not the bot ever told you its name. In the contracted one, licences, paid APIs, listings and partnerships connect parties who know who they're dealing with. The crawled internet is bigger and will stay bigger, but the contracted one is where you get paid, credited and linked.

Most SaaS companies live entirely in the first without ever deciding to. Open your robots.txt and your Cloudflare dashboard tonight. Which internet did you choose, and which one did your defaults choose for you?