Web and Forum under heavy traffic from AI bots

[ErosOlmi]

Since last Friday thinBasic web site and forum were under heavy traffic from some AI bots and some specific countries.
Thousands and thousands concurrent connections were blocking the web hosting plan we are

thinBasic provider, InMotionHosting, was obliged to de-active thinBasic account in order to preserve shared server functionalities.
As suggested from InMotionHosting tech dept., I put in place a cloudflare.com free account service in order to have some options to stop this traffic.

At the moment ...

1. most of the AI bots are blocked
2. connections from the following countries are blocked: Brazil, Bangladesh, India, Saudi Arabia, Iran, Vietnam.
3. Connections from US and UK are "challenged" by CloudFlare

I will see next days how it will evolve

Eros

[ErosOlmi]

CloudFlare is doing a fantastic job in protecting thinBasic web site and forum even with a Free plan.
thinBasic Forum was impossible to remain UP with so many connections and AI bots were using all resources of thinBasic web hosting plan.

In this initial phase I was very restrictive in order to reduce concurrent connection sessions from more that 5K to less than 500
Next days I will start step by step to open again few connections from AI bots.
At the end of the day ... they are collecting info that can be useful to people using AI and to thinBasic project too.

[EricE]

What is the purpose of these attacks?

[ErosOlmi]

Are not real attacks the one from AI
Are a way to create as much data ingestion as possible for LLM Ai.

Big tech player are creating immense databases of data to instruct any sort of AI
And to do that they scan with so much bot that web sites like thinBasic one go down mainly because we pay for a certain volume of traffic but AI bot creates 1000x times the traffic.
In thinBasic case we had thousands of bot from different part of the world

Now CloudFlare is in front of thinBasic web site and filtering the traffic killing connections from AI Bot that I decided to filter like Apple that is the most killing this site

[justin045]

Are not real attacks the one from AI
Are a way to create as much data ingestion as possible for LLM Ai.

Big tech player are creating immense databases of data to instruct any sort of AI
And to do that they scan with so much bot that web sites like thinBasic one go down mainly because we pay for a certain volume of traffic but AI bot creates 1000x times the traffic.
In thinBasic case we had thousands of bot from different part of the world

Now CloudFlare is in front of thinBasic web site and filtering the traffic killing connections from AI Bot that I decided to filter like Apple that is the most killing this site

If these bots were really intelligent, they could realize that they disturbs web sites they are mining in.

Justin