Canadian News Sites Are Locking AI Out While Banks and Governments Leave the Door Open

Picture two neighbours on the same street: one bolts every window while the other leaves the front door wide open. That is, roughly speaking, how Canadian websites are treating AI crawlers right now.

A study released on October 20 by Chateau Mojo, a Quebec-based SEO and AI search consultancy, looked at the robots.txt files of 388 of Canada’s best-known websites. Seven in ten Canadian news websites now ask AI crawlers to stay away. Meanwhile, not one of the country’s banks, insurers or fintechs blocks an AI crawler, and government sites are nearly as open.

Robots.txt is the small public file a website uses to tell automated crawlers which parts they may read. It carries no legal force, but well-behaved crawlers are expected to respect it, and it has become the main battleground in the fight over what AI systems are allowed to learn from.

What the study actually measured

Jeremy McWilliams, founder of Chateau Mojo, had his team examine the websites on October 17, covering news, banking, government, retail, telecom, education and five other sectors. Of the 388 sites, 372 robots.txt files could actually be read, and 21% of them block at least one AI crawler from their entire domain.

That average hides the real story. Among 93 news sites, 65 block AI crawlers, which is where the 70% figure comes from. In every other sector combined, the number sits at just 5%. A block here means the crawler is barred from the entire domain, not just a section, and the figures reflect what sites ask crawlers to do, not what crawlers actually do. The real number keeping AI out is probably higher, not lower.

French-language newsrooms draw the hardest line

One of the most interesting findings is linguistic. French-language news outlets are the strictest blockers in the country. According to the study, 86% of French-language news sites block at least one AI crawler, and 82% block the most widely used AI training crawler. Among English-language outlets, only 39% do the same.

The study does not explain the gap, but Quebec’s media market has its own ownership structures and licensing debates, and French-language publishers may see their scarcer content as more vulnerable to uncompensated absorption into training data. Whatever the reason, if you publish in French in Canada, the odds are overwhelming that you have told AI crawlers to keep their distance.

Banks, governments and the welcome mat

At the other end of the spectrum sit the sectors that have chosen not to block anything. None of the 29 banks, insurers and fintechs in the study blocks an AI crawler. Sixty of the 63 federal, provincial and municipal websites studied block none either. One provincial government, which blocks 11 different AI crawlers, stands out as the clear exception.

This is not necessarily carelessness. For most businesses, being found is the whole point of having a website. “Most other businesses want to be found, and an AI assistant can’t recommend a company whose website it isn’t allowed to read,” McWilliams said. A bank that blocks AI crawlers opts out of a growing channel of customer discovery. News publishers face the opposite incentive. Their words are the product, and every article an AI can read for free is an article the publisher did not get paid for. “Blocking can make sense for publishers, because their words are the product.”

Not every publisher is blocking, and not every block works

Publishers are not acting as a bloc. One Quebec media group blocks AI training crawlers while deliberately leaving AI search crawlers open, staying visible in AI-powered search while keeping its text out of training datasets. One national newspaper chain points 19 of its papers to a machine-readable content licence: you may read this, but only under terms we set. Two major broadcasters block no AI crawlers at all.

Some blocking is accidental or outdated. Nine sites, several of them major broadcast newsrooms, block crawler names the AI company no longer lists, while leaving its current crawler unruled. The site believes it is blocking AI when it is blocking nothing at all. “Plenty of sites are shutting AI out by accident through old templates and firewall settings,” McWilliams said.

Another finding cuts close to home for Canada’s AI industry. Sixteen sites block the crawler of a Canadian AI developer that received a $240 million federal funding commitment in 2024, including several of the country’s largest national newsrooms. Even a homegrown company backed by public money is getting the same cold shoulder as the foreign giants.

Why this matters beyond Canada

Canada is a useful early indicator: small enough to survey in full, diverse enough to matter. What its websites do now is a preview of the decisions publishers everywhere will keep making.

McWilliams put it sharply: “When most of Canada’s journalism asks AI to stay out while banks, retailers and governments let it in, what AI tools can freshly read about the country tilts toward the people selling something.” The AI assistants of the near future will have fresh, detailed knowledge of Canadian banking products and government services, and a much thinner grasp of Canadian news. That is a choice with consequences for what these systems know.

The full dataset, which names every site and every crawler, has been made free to use with credit to Chateau Mojo. In a debate that usually happens through lawyers and press statements, here is a small consultancy putting the raw evidence on the table.

Frequently Asked Questions

What is robots.txt and how does it block AI crawlers?

Robots.txt is a small text file a website places at its root to tell automated crawlers which parts of the site they may access. Site owners can name specific AI crawlers and ask them to stay out. The file is a request, not a legal barrier, but reputable crawler operators are expected to honour it.

Why do Canadian news sites block AI crawlers?

If an AI system can read and learn from every story for free, the publisher bears the cost of reporting while the AI company captures the value. The study found 70% of Canadian news sites now block AI crawlers, with French-language newsrooms at 86%.

Do banks really leave AI crawlers unblocked?

None of the 29 Canadian banks, insurers and fintechs examined blocks an AI crawler in its robots.txt file. The likely reason is commercial. Businesses that depend on being discovered have a strong incentive to stay readable to AI assistants, since an assistant cannot recommend a company whose website it is not allowed to read.