Photo of DeepakNess DeepakNess

I'm NOT Blocking AI Crawlers, For Now

During the middle of last year, Cloudflare introduced a new button called "Block AI Bots", designed to block all AI crawlers. It’s available even on their free-tier plan, and you can enable it with just one click.

Cloudflare Block AI Button

I love it! But I'm not using it.

While I think it’s a great feature, I’ve decided not to enable it on my websites—and I have good reasons. AI assistants like ChatGPT are already driving a significant amount of traffic to my projects. For example, the website shown in the Google Analytics screenshot below receives around 450 visits per month from ChatGPT alone, and I’m happy with that.

The future of AI and content

Looking ahead, I believe avoiding new technologies will only hurt us rather than help us. AI is one such technology that we shouldn’t resist—we should embrace and adapt to it.

However, if you create content that takes a lot of effort and you want to keep it exclusive to your audience, then blocking AI bots might be the right choice for you.

There’s also an ethical concern that if AI companies are "stealing" copyrighted content without permission, that’s a serious issue. We need a proper system where authors receive due credit if AI models use their work.

For now, I’m allowing AI crawlers because they benefit me. But if I find out in the future that they’re using my content without proper attribution, I won’t hesitate to block them.