Cloudflare’s Pay-Per-Crawl: Sustainable Income or Just Spare Change?
Download MP3(00:00)
OK. we're talking about ⁓ Cloud Flares new product to block, allow, or monetize AI crawlers to a publisher's website. Yeah. What does that mean? Yeah, it's complete chaos, if you ask me, in terms of how they're going to implement that. But besides the point, will it actually matter to publishers? Probably not.
(00:24)
the vast majority probably won't see much from this since Cloudflare will just be the one gating the access and determining the value because they control most of the web's DNS traffic. it's maybe a step in the right direction as far as like a model or how other publishers could do it themselves. Like you don't need Cloudflare. maybe this is the start of a product for publishers that
(00:53)
will be developed by not by us, but by someone else that will allow publishers to gain access to their crawls. ⁓ You have a lot of AI crawlers coming to the market. They're mushrooming by the thousands for different specific applications. But let's focus on Chat GPT because they are really the
(01:19)
the king of the hill, they'll likely hold on to their number one position because their brand is just so strong right now. that's from just a marketing perspective. Like once you establish a brand, unless you totally screw it up badly, you'll kind of maintain that position. And they're doing a good job. so OK, ChatGPT crawls local news publisher in Austin, Texas. And Cloudflare says, hey. ⁓
(01:47)
you can have access to this content, you have to pay. So then there's a signal back to ChatGPT that says, yeah, you can grab this for sure, but it's going to cost, and whatever the cost is. And then somebody has to make a decision, probably a human, I would imagine, at ChatGPT that says, yeah, this is valuable. The price is fair. Or we want this.
(2:15)
but the price is too high. So there has to be a signal back saying, you know, we're going to negotiate and do this. So I think it's doable. I like it. I think it's a super interesting idea, but is it falling under kind of the micro transaction umbrella? For sure. For sure. It's like, it won't be anything meaningful to publishers, especially the local news publishers because like,
(2:40)
You know, even the best local news publishers are lucky to see a million page views per month, like even in like decent size local news markets. So like as a million page view site going to see much action from chat GPT anyway. Right. I don't know. Depends, I guess on the topic, but probably not a ton. And then there's like, what like does cloud flare?
(03:10)
pay out this like dividend at the end of the month? Is it some quarterly payout that they do or is it some access to free cloudflare services because you've allowed access to the AI bots? Like who knows what they'll come up with, but yeah, it's in the same kind of vein as these micro transactions that probably won't do much, won't be meaningful movement for the publisher.
(03:35)
I look at Netflix and I think, okay, so I want to watch a Netflix show. And I was like, okay, ⁓ I need to pay, you know, 15 bucks a month or whatever it is these days to watch Netflix. I don't want to do that. So I'm going to go to ChatGPT and just, and just, you know, chat up the Netflix show that I want to watch because maybe ChatGPT has a deal with Netflix so that it can show me the show in ChatGPT. I don't think that's going to happen. Right? I feel like...
(04:06)
And publishers like should equate their content to a Netflix subscription because that's what it is. If you're doing a really good job and we have publishers that are just killing it in local markets, making good digital ⁓ content revenue because their content is valuable and people want it. I think that in the, what the advertising world has taught us is that digital advertising is not great as far as revenue goes and paid subscription revenue.
(04:35)
absolutely blows ad revenue, general like Google ads, kind of ad revenue out of the water. For sure. Not sponsor revenue, but automated network ad revenue. So it kind of doesn't make sense to basically knock out your best paying customers, your readers, because they will go away if you're just going to get chat GPT to ingest all your content. They'll just go there.
(05:05)
So I think what's actually happening is what we're seeing happening now with some publishers where they're just blocking down their content, period. People stop and saying, look, you can register for free. I'll give you a little more content. Or you can pay and get everything. But essentially, all the AI crawlers, all the Google crawlers, everything is just blocked. And to me, that's part of the journey of the local publisher is going from maybe I start with print, maybe print goes away. I focus on digital. I'm building my audience, my email list.
(05:35)
big time through free registration. I'm locking it all down so the crawlers don't get in and I'm requiring paid payment for great content, which is the Netflix model, right? Yeah, yeah, for sure. Yeah. So, all right. I really thought this was a cool idea, but now that we've chatted about it, I feel like this kind of, there's just not enough money there where the
(05:59)
reader will give you a lot more money for your content. You know, only if
(06:07)
whenever chat GPT makes a request to your site, or you can then require them to provide an email address. Ooh, now you're thinking. So send that back to the user. We need an email address. It's got to be a legitimate one, perhaps the one inside of your chat GPT account. And then we can get the data from that site and we'll put you on the newsletter. Fine. Have at it.
(06:33)
I like that idea actually quite a lot because that's more valuable than micro payments. sure. Micro payments only matter to sites that are getting like hundred million page views. And even then they're probably not that concerned about it because they're making money in other ways. Yeah, for sure. All right. Well, that ⁓ was a quick pod. Tyler, good one. All right, till next time.
