Cloudflare Setup
-
Hi all,
We’ve been using Cloudflare for a while now and atm we are in the free tier if that helps(and have turned off acceleration).
Since like last week we’ve seen a sudden hike in requests to the point where the server cannot handle it anymore.
These should be AI crawlers and we’re trying to limit access to them but they seem to crawl into the forum somehow.
While we would want non-logged-in users/visitors to see the content, since no one can use the site anymore we have decided to only let in logged-in users without checks and for others, for a human check to go through.
The following is the rule we’re using to identify logged in/actual users:
I was wondering if there is a better/more standardised/more accurate way to identify logged in users.
Thanks very much!
-
Hi all,
We’ve been using Cloudflare for a while now and atm we are in the free tier if that helps(and have turned off acceleration).
Since like last week we’ve seen a sudden hike in requests to the point where the server cannot handle it anymore.
These should be AI crawlers and we’re trying to limit access to them but they seem to crawl into the forum somehow.
While we would want non-logged-in users/visitors to see the content, since no one can use the site anymore we have decided to only let in logged-in users without checks and for others, for a human check to go through.
The following is the rule we’re using to identify logged in/actual users:
I was wondering if there is a better/more standardised/more accurate way to identify logged in users.
Thanks very much!
Since you are on Cloudflare, do you have their Block AI bots rules on?
They seem to help. -
Since you are on Cloudflare, do you have their Block AI bots rules on?
They seem to help.D1re_W0lf those rules do not seem to help, I have enabled them as well.
yasas we have been seeing the same behaviour starting perhaps a week ago. The bursts seem to happen for maybe half a day and then disappear.
Since we wanted the site to start working quickly, we opted for an allow-list approach. We turned on “I’m under attack” mode, which has some unfortunate side effects (namely, causing federation to stop).
These are our rules:
(any(http.request.headers["accept"][*] eq "application/ld+json; profile=\"https://www.w3.org/ns/activitystreams\"")) or (any(http.request.headers["accept"][*] eq "application/activity+json")) or (http.request.method eq "POST" and starts_with(http.request.uri.path, "/inbox")) or (starts_with(http.request.uri.path, "/assets")) or (starts_with(http.request.uri.path, "/.well-known"))This lets (respectively):
- ActivityPub fetches
- Same
- ActivityPub publishes to the NodeBB inbox
- Static assets
- Certbot and webfinger (also for AP)
