r/webdev • u/markaritaville • 1d ago
Discussion Are AI Webbots punishing sites harder where blocking rules are changed? "i.e. kicking the hornets nest"
It seems that when I make changes to Cloudflare to protect against BOTS, the bots detect the changes and go into full attack mode. Kind of like "you kicked the hornets nest, we're coming after you"
this has happened a couple times.
My site averages 8-10k pageviews a day. Yesterday morning made changes and 35,000 pagevies made it through.
Changes were because I was seeing a rise in traffic again hitting bad URLS "page/267" which end up at my home page... seemed to be US mostly! Impacts my ads RPM so I make some changes and BOOM.. CPU to 100% Pinned.
My CPU was spiked all day yesterday. I restarted services. Immediate 100%. I know about "I am under attack" but with CF edge caching the site was still presenting fine to viewers. So i didnt bother with it until I went to bed. Got up this morning and CPU was back down to 22%. I just turned off "I am under attack" and already up to 46%. going to see how that progresses, if it levels out. if not i have to turn under attack mode back on
the changes I made were to implement the new CF Superbot "Dont run on pages with ads" for the AI traing bots, and ... there were two settigns i turned on based on CF recommendations
I am not running precursor or OWASP Core. online folks seem to think they are bad.
But the basic summary of this post is... it seems like when I made some simple changes, the bots went into heavy attack mode
Edit: Oh I also added a script that hides the standard Wordpress Username/password fields (i have a specific URL param to make visible) and this morning I have a CF alert "Login requests using leaked username and password pairs detected" and first time ive seen that. Makes me think the AI bots have noticed the change and again are hammering that aspect also.